What Is Loop Engineering? AI Workflows That Improve Themselves
Loop engineering is the term I use for designing an AI workflow as a loop instead of a one-off prompt. The workflow runs, its output is checked against an eval or pass/fail rule, the result is logged, failures are fed back into the prompt, skill or data, and the whole thing reruns on a schedule.
Key points
Prompt engineering shapes one instruction. Context engineering shapes everything the model sees. Loop engineering shapes how a workflow gets better between runs.
Every loop has six parts: a trigger, a task, an eval, a log, a fix step and a schedule.
The eval is the part most teams skip, and without it a workflow cannot tell a good run from a bad one.
Fixes go to the prompt, a reusable skill, or the data in the shared knowledge base of an AI operating system.
Start with one workflow, ten to twenty pass/fail checks, and a person approving every fix.
Where does loop engineering fit after prompt and context engineering?
It is the next step after the two practices most teams already know. Prompt engineering improves the instruction, context engineering improves what the model sees, and loop engineering improves the workflow over time using its own results.
Zapier's explainer on what prompt engineering is defines it as "the process of carefully crafting prompts (instructions) with precise verbs and vocabulary to improve machine-generated outputs in ways that are reproducible."
Anthropic's engineering team moved the frame wider in its post on effective context engineering for AI agents. It says context engineering "refers to the set of strategies for curating and maintaining the optimal set of tokens (information)" and that the team views it "as the natural progression of prompt engineering." The same post warns that context "must be treated as a finite resource with diminishing marginal returns."
Both practices are about getting one run right. Neither says what happens on run 50, after the inputs have drifted and nobody has checked the output in weeks.
How do prompt, context and loop engineering compare?
The cleanest test is to ask what you are designing and how you know it worked. Each practice answers those two questions differently.
Prompt engineering | Context engineering | Loop engineering | |
|---|---|---|---|
What you design | The instruction: wording, examples, output format | Everything in the model's window: instructions, retrieved docs, tool results, memory | The cycle around the workflow: trigger, eval, log, fix and schedule |
Unit of work | One prompt | One run of an agent or workflow | Many runs over weeks |
Typical failure mode | Vague or ambiguous instructions produce inconsistent output | Too much, too little or stale context, so the model misses what matters | No eval, so quality drifts and nobody notices until a customer does |
How you measure | Read a handful of outputs by eye | Check whether the model used the right information on a test case | Pass rate on a fixed eval set, tracked run over run |
The columns stack. A good loop still needs a clear prompt and clean context, and the loop tells you which one to fix next.
What are the parts of a loop?
A loop has six parts: trigger, task, eval, log, fix and schedule. If any one is missing, the workflow either never improves or improves in ways you cannot see.
Trigger
The event that starts a run: a form fill, a new row, an incoming email or a clock. Relevance AI's platform introduction describes the setup step as configuring "escalations, approval workflows, and triggers, then put your workforce to work."
Task
The work itself: a prompt, an agent or a chain of automation steps.
Eval
A check that decides whether the output passed. OpenAI's evals guide says evals "test model outputs to ensure they meet style and content criteria that you specify."
Log
A record of each run: input, output, eval result, and which prompt or skill version produced it.
Fix
A person or a second model reads the failed runs, finds the pattern and edits the prompt, a skill or the source data.
Schedule
The rhythm that reruns the evals and reviews the log. Daily for high-volume workflows, weekly for most others.
Without a loop, failures quietly repeat. A source doc changes, the agent keeps reading the old version, and the drift is only caught after the output ships.
What counts as an eval in a loop?
An eval is any repeatable check that returns pass, fail or a score for an output. It can be code, a second model grading against a rubric, or a person.
Anthropic's post on demystifying evals for AI agents defines an eval as "a test for an AI system: give an AI an input, then apply grading logic." It notes that agent evaluations "typically combine three types of graders: code-based, model-based, and human."
Each grader type has a trade-off:
Code-based graders. Anthropic describes them as fast and cheap, but "[b]rittle to valid variations that don't match expected patterns exactly." Good for format, required fields and length.
Model-based graders. Useful for judgment calls like tone or relevance. The post notes they are non-deterministic and require calibration with human graders for accuracy.
Human graders. Accurate but slow and expensive. Use them to calibrate the other two.
You do not need hundreds of test cases. The same post says "20-50 simple tasks drawn from real failures is a great start."
n8n's evaluations overview splits this into two stages. Before launch, "it often makes sense to generate just a handful of examples" and compare results by eye. After launch, datasets are built from real executions and scored with a numeric metric so you can track quality across runs.
That second stage is the loop: the metric tells you whether last week's fix helped.
How do you turn one workflow into a loop?
Pick one workflow that already runs, write down what good output looks like, and add the missing parts one at a time. The numbers below are illustrative and hypothetical.
The example workflow
Illustrative example: a B2B team uses an AI workflow to draft first replies to inbound demo requests. A person reviews each draft before it sends. Volume is about 50 requests a week, and reviewers rewrite roughly half the drafts.
The eight steps
Pick the workflow and its owner. Choose one workflow with steady volume and one person accountable for it: here, the sales lead who reviews drafts.
Write the pass/fail rules. Turn the reviewer's edits into checks: names the company correctly, mentions the right offer for the stated team size, includes one meeting link, stays under 120 words, and avoids banned phrases.
Build a starter eval set. Pull 20 past requests where the draft was rewritten and 10 where it was sent as is, each with the approved reply.
Score the current version. Run the 30 cases and grade them. Code checks handle length, links and phrases. A model grader checks offer fit, and a person spot-checks its grades.
Log every run with a version number. Store input, output, each check's result and the prompt version in one Notion table or sheet.
Find the pattern in the failures. Group the failed checks. If most fail on offer fit, the cause is likely the data (an outdated pricing page), not the prompt.
Fix in one place and rerun. Change one thing, rerun all 30 cases and compare the pass rate to the last version.
Put it on a schedule. Each week, add rewritten drafts to the eval set, rerun the suite and review the log. A fix ships only if the pass rate rises and no older case fails.
After a few weeks the reviewer spends time on new failures instead of the same ones.
Where should a fix go: the prompt, a skill or the data?
Put the fix where the failure started. Wording problems belong in the prompt, repeated know-how belongs in a skill, and wrong facts belong in the data.
Failure pattern | Likely source | Where to fix it |
|---|---|---|
Output ignores a format or tone rule | The instruction | The prompt |
The same mistake appears across several workflows | Missing shared know-how | A reusable skill |
Output is well written but factually wrong | Stale or missing source material | The data or knowledge base |
Output is right but arrives too late or too often | The trigger or schedule | The loop settings |
Skills matter most once you run several loops. Gumloop's post on syncing skills between GitHub and Gumloop says skills "teach agents how your company works" and give agents "reusable knowledge that they can apply at scale."
The same post describes the control that makes self-editing safe: skill changes "get reviewed and approved before reaching your agents," and commits show "who changed what, when, and why." That is the fix step with a review gate and a log built in.
The same review-and-fix rhythm works outside sales. Here is an illustrative content workflow run as a weekly loop: extract, map, accelerate, measure, structure, draft and publish.
How much should a loop fix itself without a person?
Less than you think at first. Let the loop propose fixes and a person approve each one, then hand over more as the pass rate holds.
Anthropic's evals post describes regression evals that "should have a nearly 100% pass rate" and "protect against backsliding." That makes a good gate: a model proposes an edit, reruns the suite and ships only if every old case passes.
Some fixes should always stay with a person:
First contact with a customer. A reply to a new prospect, a quote or a contract term.
Pricing, offers or policy. These belong in the knowledge base, edited by their owner.
Changes to the eval itself. If the loop can loosen its own rules, the pass rate stops meaning anything.
The L1 to L4 AI automation maturity ladder lays out how to move a workflow from full review to light review in stages. Access rules matter just as much, and the guide to rolling out Claude to a team securely covers who can edit what before loops start changing prompts.
What are the common mistakes in loop engineering?
The most common mistake is adding a schedule without an eval. The workflow reruns daily and never improves, because nothing marks which runs failed.
No written definition of good. If the reviewer cannot list the rules, the eval cannot check them.
Grading with a model you never calibrated. Compare its grades to a person's on the same cases first.
Changing several things at once. You cannot tell which change moved the pass rate.
Fixing data problems in the prompt. Telling the model "our price is now X" in the prompt leaves every other workflow reading the old number.
Letting the loop edit its own eval. The eval is the fixed point. Changes to it should go through a person.
No log or version numbers. Without them, a drop in quality has no trail back to its cause.
What tools can you build a loop on?
Any stack that can trigger a run, store results and edit a prompt will do.
Trigger and schedule. Zapier, n8n, Gumloop or Relevance AI triggers, or a scheduled task in Claude.
Task. Claude or another model through OpenRouter, often inside an agent built in Relevance AI, Gumloop or n8n.
Eval. Code checks in the automation tool, a model grader prompt, n8n's built-in evaluations, or OpenAI's evals.
Log. A Notion database or a sheet with one row per run.
Fix. Edits to prompts and skills, ideally versioned. Gumloop's GitHub sync is one example of that pattern.
Review. A Slack message to the workflow owner when the pass rate drops or a fix is proposed.
I build loops on Notion, Claude, Relevance AI, Composio, Slack and OpenRouter, and keep the current list on my stack page. If you are still choosing between rules, agents and a shared layer, start with the difference between an AI operating system and AI agents.
What is a loop worth in dollars?
A loop does not create new savings by itself. It protects the savings a workflow was built for, because a workflow that drifts ends up back in manual review.
Here is the worked example from my AI Workflow Install page: a 12-person team where each person spends 8 hours a week on manual work, at a $60 loaded hourly rate. That is $299,520 a year, and in the example one workflow accounts for half of it, $149,760.
If that workflow drifts until people rewrite half its outputs, much of that value leaks back out. The loop keeps it from drifting.
To find the workflow worth protecting first, the manual-work leak audit guide shows how to time each one and rank them by yearly cost.
FAQ
Is loop engineering the same as prompt engineering?
No. Prompt engineering designs one instruction. Loop engineering designs the cycle that tests the workflow's output, logs the result and decides what to change next, which often includes the prompt.
Do I need a coding background to build a loop?
No. Tools like n8n, Gumloop, Zapier and Relevance AI handle triggers and schedules without code, and a Notion database or sheet works as a log.
How many test cases does an eval set need?
Anthropic calls 20 to 50 simple tasks drawn from real failures a great start. Add new failures each week.
Can an AI workflow really improve itself?
It can propose and test its own fixes against an eval set. Have a person approve fixes until the regression suite passes reliably, and never let the loop edit its own eval.
How often should a loop run its evals?
After every change to the prompt, model or data, plus on a fixed schedule. Weekly suits most business workflows; daily suits high-volume ones.
Which workflow should I turn into a loop first?
One that already runs, has steady volume and has a reviewer who edits its output. Their edits become your first pass/fail rules.
Find out which workflow to turn into a loop first
If you want help picking the workflow and writing its first eval, book a free 30-minute Leak Call. We estimate what manual work in one area costs per year, and you get a costed workflow list within 48 hours. You can also see all my offers on the services page.
About the author: Kevin Payne is an AI engineer and GTM operator who installs AI workflows for B2B teams. He has 10+ years in B2B SaaS demand generation and RevOps across Techstars, a16z and YC portfolio companies, and his writing has appeared in HubSpot, GetResponse, Outbrain and more.
