The decision between building an AI agent and orchestrating a deterministic prompt workflow is the most consequential architectural choice in 2026 AI engineering, and the one most commonly made on vibes. Agents are fashionable. Workflows are correct most of the time. The four variables that should drive the decision are cost (agents are 3 to 10 times more expensive per task), debuggability (agent regressions have a much larger surface of possible causes), latency (workflow latency is predictable; agent latency varies wildly), and search-space size (agents pay off when the task has too many paths to enumerate, and not before). This piece names the frame, names the threshold at which an agent starts beating a workflow, and explains why the modal production system in 2026 should be a static DAG of prompt calls rather than a ReAct loop. This is the decision a sourcing-conscious engineering org has to make before it ever touches build-vs-buy on the framework underneath.
This is a spoke under the AI build-vs-buy-vs-hire decision matrix for 2026. The matrix’s principles assume a capability stack to source against; this piece operates one level up, deciding whether the capability you are sourcing should be an agent at many or whether it should be a workflow that can be sourced more cheaply and shipped more reliably.
The terminology that prevents the decision from happening
A workflow is a static DAG of prompt calls; the steps and their order are decided at design time, encoded in code, and executed deterministically. The model may make local routing decisions (branch A or branch B from a known set), but the decision space is bounded and the control flow is known.
An agent is a control loop in which the model decides at runtime which step to take next. The canonical pattern is ReAct (Reason, Act, Observe): the model writes a reasoning trace, calls a tool, observes the result, and repeats until it decides the task is done. The model owns the control flow; the engineer owns the tool registry and the stop conditions.
Workflows are predictable; agents are flexible. That sentence is the entire trade-off, and most teams resolve it on the wrong side because “agent” sounds modern and “workflow” sounds like 2018. The terminology hides the engineering content. The four variables below put the engineering content back in front of the decision.
Variable 1: Cost
Agents make multiple LLM calls per task. A typical ReAct agent makes 4 to 14 calls to complete a task that a workflow would complete in 2 to 5 calls. Each call passes the accumulated context; prior reasoning, prior tool outputs; which inflates input tokens at most step.
Empirically, agent solutions cost 3 to 10 times more per completed task than equivalent workflows for the same problem. The variance is wide because it depends on the planner’s efficiency, the tools’ speed, and the model’s tendency to take exploratory steps. The high end of the range is what shows up when the agent gets stuck in a loop and the engineer adds a step-count cap that papers over the loop without fixing it.
For high-volume tasks; anything north of 10K completions per day; the cost gap is the dominant variable. A workflow that costs $0.04 per completion at 100K daily completions costs $4K daily; an agent at $0.40 per completion costs $40K daily. Over a year that gap is $13M, which buys a lot of engineering time on a workflow that does the same job. The accounting is covered in detail in decoding cost-per-query as a defensible unit-economics framework.
For low-volume, high-value tasks; research, debugging, ops automation at human cadence; the cost gap is real but not dominant, because the volume is small enough that other variables decide.
Variable 2: Debuggability
A workflow has known control flow. Debugging means inspecting the inputs and outputs of each fixed step. The cause space for a regression is small: a prompt changed, a tool changed, the model version changed, the input distribution changed. Each of those is detectable with standard observability and addressable with standard fixes.
An agent has model-decided control flow. Debugging means reconstructing why the model picked the path it picked, which is non-deterministic and changes silently with model versions. An agent regression can be a model-version regression, a prompt regression, a tool regression, or a planner regression; and the symptoms are often indistinguishable from the surface. “The agent stopped resolving customer issues correctly” might mean any of four upstream changes, and the diagnosis is a multi-day investigation rather than a multi-hour one.
The debuggability gap compounds operationally. A team running 30 production workflows can be on-call with two engineers. A team running 30 production agents needs more on-call capacity because most incident takes longer to diagnose. The on-call cost is rarely budgeted at the agent-vs-workflow decision and shows up six months in, when the team realizes they cannot keep up with the incident queue.
Variable 3: Latency
Workflows have predictable latency; the sum of the fixed steps plus whatever parallelism the DAG permits. P50 and P99 are close together, because the path variation is small.
Agents have latency that varies with the planner’s decisions. An agent that takes 4 steps for one query and 14 steps for another has a P50 of perhaps 12 seconds and a P99 of 45 seconds. For interactive workloads, the variance is often more painful than the median; users tolerate slow once but not unpredictable. The agent that runs in 4 seconds for the demo runs in 38 seconds for the long-tail customer issue, and the customer’s experience is “this thing is broken” rather than “this thing is slow.”
The fix that teams reach for; capping the agent’s step count; addresses latency variance at the cost of correctness on the long tail. A capped agent that hits the cap returns a half-finished answer that looks confident, which is worse than the workflow’s deterministic but bounded answer. The cap is a workflow-shaped constraint applied to an agent runtime.
Variable 4: Search-space size
The variable that should decide the architecture, and the one most often skipped.
The question is: how many distinct paths does the task have? If the answer is small (10 to 50 well-known paths), a workflow plus routing prompts at branch points handles many of them. If the answer is large (hundreds of paths, or unbounded because the task involves arbitrary websites or arbitrary codebases), the workflow becomes unmaintainable and an agent’s runtime planning is doing real work.
A research agent navigating arbitrary websites cannot be a workflow because the path space is open. A customer-support agent handling open-ended issues, where each ticket might require a different sequence of internal-system queries, has a path space too large to enumerate. A debugging agent exploring a codebase has unbounded paths.
A customer-support agent handling 30 known issue types is bounded; it should be a workflow with 30 branches and a routing prompt. A document-processing agent handling 12 document types is bounded. A data-extraction agent over a known schema is bounded. The path space is the deciding variable, not the surface complexity.
The diagnostic: if the team can list most distinct path the agent will take in a 4-hour whiteboard session, it is a workflow. If the team cannot, it is an agent. Most production tasks fail the listing test in the first hour because the team has not mapped the path space; the right move when that happens is to spend two more weeks mapping rather than to declare the task an agent task by default.
The decision rule
Combine the four variables into a single rule that resolves most cases:
- High volume + low path variability (most enterprise workloads) → workflow
- High volume + high path variability (rare in enterprises, common in agentic-product startups) → agent, with explicit eval and on-call investment
- Low volume + low path variability → workflow (faster to ship, cheaper to run, easier to debug)
- Low volume + high path variability (research agents, debugging agents, ops automation) → agent
The modal production AI system in a 2026 enterprise is high volume + low path variability. The right architecture for that quadrant is a workflow. Most teams get the architecture wrong because they assume “AI” implies “agent,” and pay 3-to-10x cost premium plus the debuggability and latency penalties for an architecture they did not need.
ReAct vs static DAG: a worked comparison
A single customer-support task built both ways: classify incoming email, retrieve policy documents, draft response, send or escalate.
Static DAG: four nodes. Classify into 1 of 12 issue types (routing prompt). Retrieve documents (parameterized). Draft response (template-aware prompt). Send or escalate based on confidence threshold. Total: 3 LLM calls. P50 latency 4 seconds, P99 8 seconds. Cost $0.06 per task. Eval is 4 deterministic test points per branch.
ReAct agent: single agent with tool registry (classify, retrieve, draft, send, escalate). Average 7 LLM calls per task. P50 latency 18 seconds, P99 60 seconds. Cost $0.42 per task. Eval is harder because path is variable.
For 100K tasks/day: workflow costs $6K daily, agent costs $42K daily. Workflow ships in 6 weeks; agent ships in 14 weeks because the planner needs tuning. The agent is the wrong architecture: path space bounded (12 issue types), volume high. The workflow wins on cost, latency, debuggability, shipping speed. The agent wins only on flexibility the team does not need. The composition is described in stop building AI plumbing, buy the rails, build the moat.
Frequently asked questions
What is the difference between an agent and a workflow?
A workflow is a static DAG of prompt calls; steps and order decided at design time, encoded in code, executed deterministically. An agent is a control loop in which an LLM decides at runtime which step to take next, often via ReAct or a planner pattern. Workflows are predictable; agents are flexible.
Why are agents more expensive than workflows for the same task?
Agents make multiple LLM calls per task, re-passing context at each step. Empirically, agent solutions cost 3 to 10 times more per completed task than equivalent workflows because most reasoning step is an LLM round-trip. For high-volume tasks the cost gap is the dominant variable.
Why are agents harder to debug than workflows?
Workflows have known control flow. Agents have model-decided control flow that changes with model versions. A workflow regression is a code regression. An agent regression can be model-version, prompt, tool, or planner regression. The cause surface is much larger.
How does latency differ?
Workflow latency is predictable. Agent latency varies with the planner’s decisions; P50 of 12 seconds, P99 of 45 seconds is common. For interactive workloads the variance is often more painful than the median.
When does an agent beat a workflow?
When the task has high path variability that cannot be enumerated at design time; research agents on arbitrary websites, debugging agents exploring codebases, support agents on open-ended issues. When the task has 10 to 50 well-known paths, a workflow plus routing prompt beats an agent on most variable.
What is ReAct and when does it pay off?
ReAct (Reason, Act, Observe) is the canonical agent pattern. It pays off when the search space is large enough that reasoning meaningfully prunes it. When the search space is small enough to enumerate, the reasoning trace is overhead.
Can a workflow include any model decisions?
Yes. Routing prompts at specific nodes (model picks A or B from a known set) keep it a workflow. The distinguishing feature is that the model’s decision space is bounded and known at design time.
Should new AI features start as agents or workflows?
Workflows. Default is a static DAG with routing prompts. Promote to an agent only when path enumeration produces diminishing returns. Starting as an agent and consolidating into a workflow later is harder than the reverse.
How does this interact with build-vs-buy?
Workflows are easier to build and easier to buy because behavior is enumerable. Agents are harder to buy because behavior is emergent. Most production systems should be workflows, which means most production sourcing decisions are workflow decisions.
What is the most common architectural mistake?
Building an agent because agents are fashionable, then spending six months guardrailing it until it functionally becomes a workflow with a more expensive runtime. Starting as a workflow would have shipped faster, cost less, and ended at the same reliability.
Key takeaways
- Workflows (static DAGs of prompt calls) and agents (model-driven control loops) trade predictability against flexibility; most teams resolve the trade-off on vibes rather than on the four hard variables.
- Cost: agents are 3 to 10 times more expensive per task. Debuggability: agent regressions have a much larger cause surface. Latency: agent P99 latency varies wildly. Search-space size: agents pay off only when paths cannot be enumerated.
- The decision rule: high volume plus low path variability (the modal enterprise workload) is a workflow; agents are correct for low-volume, high-variability tasks.
- ReAct is the right pattern only when the search space is large enough that the reasoning trace prunes meaningfully; otherwise it is overhead.
- The most common error is building an agent because agents are fashionable, then guardrailing the agent until it functionally becomes a workflow with a more expensive runtime. Starting as a workflow would have shipped faster and cost less.
The architectural choice between agent and workflow is upstream of most other sourcing decision. Get it right and the rest of the stack; model selection, framework choice, eval design, on-call structure; falls into place at the cost and reliability the business signed off on. Get it wrong and the system fights itself for two years before someone proposes a rewrite that is, structurally, the workflow that should have been built on day one.
Arthur Wandzel
