How to test an AI workflow before you trust it
Run it against a test set of real past inputs where you already know the right answer, score every output with a grader, and only switch it on when it clears a pass rate you set in advance. Then rerun the same test set every time you change the prompt, the model or the data it reads.
That test set plus its graders is called an eval. It sits in the middle of the skill stack in AI engineering for GTM operators: after you can write prompts and before you let workflows improve themselves.
Key points
An eval is a fixed set of test cases plus graders that score each output, so "it seems to work" becomes a number.
Build the test set from real past inputs (leads, tickets, CRM records) where you already know the correct result, including the messy ones.
Use code checks for anything rule-based, rubric graders for judgment calls, and human review to calibrate both.
Decide the go-live pass rate before you run the first test, not after you see the score.
Rerun the full set after every prompt or model change; a fix for one case often breaks another.
What is an eval, in plain operator terms?
An eval is a repeatable test for an AI step. OpenAI's evaluation best practices guide defines evals as "structured tests for measuring a model's performance."
Anthropic's engineering post on evals for AI agents breaks one into three parts, and the vocabulary maps cleanly onto RevOps work:
Task. "A single test with defined inputs and success criteria." In GTM terms: one inbound lead and the owner it should route to.
Trial. "Each attempt at a task is a trial." You run each task more than once because model output varies.
Grader. "Logic that scores some aspect of the agent's performance." A check that the owner field matches, or a rubric that scores an email draft.
Braintrust's eval quickstart uses a similar three-part frame: data ("a dataset of test cases with inputs and expected outputs"), a task ("an AI function you want to test") and scores ("scoring functions that measure output quality").
A routing rule gives the same answer every time; a model may not. That is why one good demo proves very little.
Why do vibe checks fail for GTM workflows?
A vibe check tests the five examples you tried. Production sends hundreds you never saw.
OpenAI lists "Vibe-based evals: Using 'it seems like it's working' as an evaluation strategy" as an anti-pattern. Hamel Husain, in his post Your AI product needs evals, makes the same case from the builder side: "you need a way to measure success beyond vibe checks."
GTM workflows make vibe checks riskier than most:
Messy inputs. Fake company names, empty enrichment, duplicate contacts.
Quiet errors. A misrouted lead throws no error. It sits with the wrong rep until it goes cold.
Frequent changes. Territories and ICP definitions shift, and each shift changes what "correct" means.
Downstream trust. Reps act on AI research notes without rechecking them.
Husain's other line matters most: "You must remove all friction from the process of looking at data."
How do you build a test set from real past inputs?
Pull 20 to 50 real past cases where you already know the right outcome, then add the edge cases that break things. Anthropic's agent evals post says "20-50 simple tasks drawn from real failures is a great start."
A six-step build for a business workflow:
Write the job in one sentence. "Given a demo request, return the correct owner." If you cannot, start with the guide on turning a process document into an automation.
Export real history. Pull recent inputs from the CRM with the outcome a human decided. Those become your expected answers.
Seed it with past failures. Every lead someone complained about, every ticket that bounced between teams.
Add edge cases on purpose. Anthropic's guide to developing test cases lists "irrelevant or nonexistent input data," overly long input, "poor, harmful, or irrelevant user input" and ambiguous cases "where even humans would find it hard to reach an assessment consensus."
Check each case is unambiguous. Anthropic's agent post sets the bar: "A good task is one where two domain experts would independently reach the same pass/fail verdict." If your ops lead and a sales manager disagree on the answer, fix the case or drop it.
Strip what should not travel. Remove personal data the eval does not need.
Store the set as a plain table: input, expected output, notes. A Notion database works; the guide to connecting Claude to Notion shows how Claude can read and append to one.
Keep growing the set after launch. LangSmith's evaluation concepts docs put it simply: "Once in production, convert real traces into examples." Every bad output a rep flags becomes a test case.
Which grader should you use: pass/fail checks, rubric graders or human review?
Use the cheapest grader that can actually judge the thing you care about. Anthropic's test case guide says to "choose the fastest, most reliable, most scalable method," and ranks them this way.
Grader | What Anthropic's guide says | GTM example | Use it for |
|---|---|---|---|
Code-based check | "Fastest and most reliable, extremely scalable," but "lacks nuance" | Owner ID equals the expected owner; output is valid JSON; no field left blank | Routing, field updates, classification, format |
LLM-based rubric grader | "Fast and flexible, scalable and suitable for complex judgment. Test to ensure reliability first, then scale." | Does the research note name the account's current tool stack? Is the email under 120 words with one ask? | Drafts, summaries, research notes |
Human review | "Most flexible and high quality, but slow and expensive. Avoid if possible." | A sales manager scores 20 call summaries | Calibrating the other two, new use cases |
Pass/fail checks
Hamel Husain notes that "Unit tests for LLMs are assertions." Write one per rule the output must follow. Most CRM-facing workflows get most of their coverage here, because the output is a field value or record ID.
Rubric graders
When the output is prose, a second model scores it against a written rubric. Anthropic's tips: "Have detailed, clear rubrics," and "Be empirical or specific," for example asking the grader to output only "correct" or "incorrect" or a 1 to 5 score. Its examples also note it is "generally best practice to evaluate with a different model from the one used to generate the evaluated output."
OpenAI flags "Position bias (response order), verbosity bias (preferring longer responses)" in model judges, and suggests "pairwise comparison or pass/fail for more reliability."
Human review
Humans set the standard. OpenAI's guide says to "use human feedback to calibrate automated scoring" and to scale a model judge once it "consistently agrees with human annotations." Have a person grade 20 outputs, run the rubric grader on the same 20, and compare.
How much review an output needs depends on two things: how sensitive the account is and how risky the action is.
What threshold should a workflow hit before go-live?
Set the bar by the cost of one wrong output, and write it down before the first run. A routing step that sends leads to the wrong rep needs a higher bar than a first-draft email a rep will edit anyway.
These thresholds are illustrative starting points, not industry standards:
Workflow type | Illustrative go-live bar | Who sees the output first |
|---|---|---|
Lead routing or CRM field updates | 98% or higher on code checks, zero failures on the past-failure cases | Nobody; it writes to the CRM |
Account research notes for reps | 90% on rubric grader, spot-checked weekly | The rep, before outreach |
Email or follow-up drafts | 85% on rubric grader, human approves each send | The rep, who edits and sends |
Customer-facing replies | Highest bar you can test, plus human review on anything low confidence | The customer |
Two rules make the threshold mean something:
Test consistency, not just one lucky run. Anthropic's post separates pass@k, which "measures the likelihood that an agent gets at least one correct solution in k attempts," from pass^k, which "measures the probability that all k trials succeed." For a workflow that runs unattended on every lead, the second number is the one to watch.
Start with a human gate. Ship at the lower rung first, with a person approving outputs, and move to full automation only after the live pass rate holds. The AI automation maturity ladder lays out those rungs.
Most revenue teams skip this step, and it is where a quiet routing error turns into lost pipeline. In the systems I build for my own work, nothing writes to a database until it clears its test set, and I do the same for SaaS revenue teams (lead routing, CRM cleanup, pipeline reporting) in a 90-day AI RevOps Sprint for Series A and B SaaS. Contact me to get yours scoped.
After go-live, the same checks become ongoing supervision: track accuracy, drift and exception rate, audit samples, and set escalation triggers.
How do you run regression tests after a prompt change?
Rerun the full test set after every change to the prompt, model, tools or context, and compare to the last run. A change ships only if it keeps every case that used to pass.
Anthropic's agent post defines the job: "Regression evals ask, 'Does the agent still handle all the tasks it used to?' and should have a nearly 100% pass rate." It also describes how capability evals with high pass rates can "graduate" into that regression suite.
A simple change routine:
Save the current prompt, model name and score as the baseline
Make one change at a time
Run the full test set, not only the case you were fixing
Compare case by case: list anything that went from pass to fail
Add the case that prompted the change as a new test
Ship only if the regression set holds and the target case passes
Braintrust says its experiments help teams "catch regressions before deployment, and validate improvements with data instead of intuition." LangSmith lists regression testing as an offline use case to "ensure new versions don't degrade quality." OpenAI's guide recommends you "set up continuous evaluation (CE) to run evals on every change."
Edits to the reference docs a workflow reads count too; the guide on what context engineering is explains why. When you create a Claude skill, keep a few test inputs beside it and rerun them after each edit.
Which eval tools fit a GTM team?
Start with a Notion table and a script. Move to a dedicated tool once several workflows or people are changing prompts.
Tool | What it says it is | Good fit when |
|---|---|---|
"An open-source CLI and library for evaluating and red-teaming LLM apps" that "runs completely locally" | You want config-file tests you can run from a terminal or CI, with no new vendor | |
Braintrust | Evals built from data, a task and scores, with experiments you compare run to run | You want a hosted UI to compare prompt versions side by side |
LangSmith | Offline evals on curated datasets plus online evals on "real production traces" | Your workflow already runs on LangChain or LangGraph |
Anthropic and OpenAI guidance | Free docs on test design and grading | You are writing your first eval and want the method before the tool |
Promptfoo's intro sums up the goal for any of these: "test-driven LLM development, not trial-and-error."
Keep the test set in a format you own, such as CSV, so switching tools does not mean rebuilding cases.
What are the common mistakes?
Most failed evals come from an unrealistic test set or a threshold set after the fact.
Testing only clean examples. OpenAI calls this "biased design": datasets "that don't faithfully reproduce production traffic patterns."
Moving the bar after the run. If you set 95% and hit 88%, the workflow is not ready.
Trusting a model grader nobody checked. OpenAI lists "ignoring human feedback" as an anti-pattern. Calibrate against human scores first.
Generic metrics. Grade what the business needs: right owner, right field, one clear ask.
Fixing one case and shipping. Without a full rerun, the fix can break three cases you stopped looking at.
Never reading outputs. Anthropic's advice is blunt: "Read the transcripts!" The score tells you how many failed; the transcripts tell you why.
FAQ
What is an eval for an AI workflow?
A fixed set of test inputs with known correct results, plus graders that score each output.
How many test cases do I need to start?
Anthropic's engineering post says 20 to 50 simple tasks drawn from real failures is a great start. Grow the set as production surfaces new failures.
Do I need to code to run evals?
Not to start. A table of inputs, expected outputs and a pass/fail column works. Tools like Promptfoo add automation later.
Should a model grade another model's output?
Yes, for judgment calls, once its scores agree with a human's on a sample, ideally using a different model from the one that wrote the output.
What pass rate is good enough for go-live?
Set it in advance by the cost of an error. CRM writes need a far higher bar than drafts a person reviews.
How often should I rerun evals?
After every change to the prompt, model, tools or context, and on a schedule for live workflows.
Want help putting evals around your workflows?
If you want a second pair of hands designing test sets and graders for the workflows you already run, see how I work or book a short intro call and we can look at one workflow together.
Running revenue at a Series A or B SaaS company? Ask about the 90-day AI RevOps Sprint. I take two clients at a time.
About the author: Kevin Payne is an AI engineer and GTM operator who installs AI workflows for B2B teams. He has 10+ years in B2B SaaS demand generation and RevOps across Techstars, a16z and YC portfolio companies, and his writing has appeared in HubSpot, GetResponse, Outbrain and more.
