An LLM agent is a language model that decides what to do next — picking from a list of tools, taking an action, reading the result, and looping until a task is done. Strip away the marketing and an agent is a planning loop bolted onto an LLM. In 2026 that loop is the difference between a chat feature and a system that closes on real work. It is also the most over-sold word in your inbox.
This piece is for a non-engineer founder who keeps seeing “agent” in vendor proposals and needs a precise definition before signing a scope. You get the 2026 definition, the four ingredients to demand by name, three signals your MVP needs an agent, three signals it does not, the eval shape, and a worked example with realistic costs. The article draws on the eval-first build playbook and the broader idea-to-product manifesto.
Table of Contents
- The 2026 definition (plain English)
- The spectrum from single call to autonomous loop
- The 4 ingredients of an agent (the buyable line items)
- 3 signals your MVP needs an agent
- 3 signals your MVP does NOT need an agent
- The eval shape for an agent feature
- A worked example: the contract-clause review agent
- Agentic-washing: 5 vendor red flags
- Frequently Asked Questions
- Closing
The 2026 definition (plain English)
An LLM agent is a system where a language model is given a goal and a set of tools, and the model itself chooses which tool to call, in what order, and when to stop. That choice — the control flow — is the load-bearing word. In regular software a programmer writes the control flow in advance. In an agent, the model writes it on the fly, one step at a time.
Anthropic made the distinction crisp in late 2024 in Building effective agents: a workflow has predetermined steps; an agent has the LLM directing its own steps. LangChain uses the same line — an agent decides the control flow of an application. The practical difference: a workflow is predictable, cheap, and debuggable; an agent is flexible, expensive, and can fail in ways the original engineer never imagined. Buy an agent when the flexibility is worth the variance — not because it is on the vendor’s front page.
The spectrum from single call to autonomous loop
“Agent” is a spectrum, and where your feature sits is the actual scope question.
- Single-call LLM. One prompt, one response, no tools. Most “AI features” still sit here.
- LLM with tools (function calling). The model can call prewritten functions — a database lookup, a calculator — then continue. One step of the loop, engineer-controlled.
- Multi-step tool-using workflow. The engineer wrote the sequence — first search, then summarize, then format — and the LLM fills each slot. Predictable.
- Tool-using agent (ReAct-style). The model decides which tool to call next based on what it has seen. The control flow is not predetermined. The pattern was named in the ReAct paper (Yao et al., arXiv:2210.03629, 2022): reasoning + acting in a loop.
- Multi-agent and autonomous agents. Higher capability, much higher variance, much higher cost. Production-rare in 2026 outside coding assistants and a few security and infrastructure use cases.
The MVP question for almost every founder is step 1 vs. step 4. Step 1 ships in a week. Step 4 needs an eval suite, observability, a cost meter, and a human in the loop. The pricing gap is usually three to five times.
The 4 ingredients of an agent (the buyable line items)
Every credible 2026 agent contains exactly four ingredients. They map onto line items in a scope of work, and a founder should be able to name all four when asked what they are paying for.
1. A planning loop. The model produces a step, observes the result, produces the next step — capped at a maximum iteration count. The ReAct loop — thought, action, observation — is the canonical shape. Often ten to fifty lines of code on top of an LLM SDK; not magic. Demand to see it.
2. A tool registry. A list of functions the model can call, each with a name, a description, and a typed input schema. Frontier providers (Anthropic, OpenAI, Google) expose this through function-calling APIs. If a vendor proposal does not enumerate the tools, the agent’s behavior is not specified.
3. A state object. A structured memory of past steps, intermediate results, errors, and the original goal. In-memory for short tasks, persisted to a database for sessions. Asking where is state kept, and what happens when it overflows the context window is a load-bearing question.
4. A stopping criterion. A rule that ends the loop — the model decides done, the iteration cap is reached, a budget is exhausted, or a human intervenes. Without one an agent can spend your money indefinitely. A hard requirement, not an optimization.
If your vendor cannot point at each of these four in their architecture diagram, you are buying a single-call LLM with a fancier marketing page.
3 signals your MVP needs an agent
Three conditions, and you need at least two of them, to justify an agent in your MVP. None of them — pay for a workflow or a single call instead.
Signal 1 — The task has variable steps. Different inputs need different tools in different orders. Research (which sources depend on the question), debugging (which file to read next depends on the error), customer-support escalation (which system to query depends on what the customer said). If you can draw the flowchart in advance and it fits on one page, you need a workflow, not an agent.
Signal 2 — The task requires real-world side effects through external systems. The agent must do more than read and write text — call an API, write to a database, send an email, edit a file. The list of effects is open-ended; the founder cannot enumerate every action in advance. Pure-text tasks rarely need an agent.
Signal 3 — The unit of work is multi-step and the LLM has to reason between steps. Find the customer’s order, see that it shipped, check the carrier’s tracking, decide whether to refund or replace. If the reasoning between steps is the value, an agent is the right shape. If the reasoning is front-loaded into a single prompt and the rest is mechanical, a workflow does it cheaper.
A practical rule: an MVP feature needs an agent when at least two of three signals fire. One alone usually means a tool-augmented single call is enough.
3 signals your MVP does NOT need an agent
Equally important — three patterns that get sold as agent work and almost never are.
Anti-signal 1 — One-shot classification, extraction, or transformation. Classify this support ticket. Extract the line items from this invoice. Single-call problems. The 2026 vendor pitch “our agent classifies your tickets” almost always means “our single LLM call classifies your tickets” dressed up.
Anti-signal 2 — Pure generation with no external dependencies. Draft a marketing email. Summarize this PDF. The model has everything it needs in the prompt. No tools, no external state, no decisions. Wrap it in an agent loop and you pay multiples for no quality gain.
Anti-signal 3 — The flow is deterministic and the engineer can write it in code. On every new lead, look up the company, generate a first email, save to CRM, schedule a follow-up. The sequence is fixed. Pay for the workflow. An agent here is over-engineering, and the variance is more likely to hurt your conversion than the flexibility is to help it.
The 2026 vendor instinct is to bid the agent shape because agents bill more. The disciplined founder bids down — start at single-call, escalate to tool-augmented call, escalate to workflow, and only escalate to agent when a signal fires and you can pay the eval bill that comes with it.
The eval shape for an agent feature
The eval for an agent is fundamentally different from the eval for a single-call LLM feature. A single-call feature is graded on output quality. An agent is graded on whether it completed the task, and at what cost. Four metrics, all required:
1. Task success rate. What fraction of attempts achieve the goal, judged by a human or a deterministic check. The headline number. Threshold set by the hallucination budget — sixty percent for a draft generator, ninety-eight percent for customer-facing work.
2. Tool-call accuracy. When the agent calls a tool, does it call the right one with the right arguments? This catches near-success — the agent queried the wrong table or sent the email to the wrong address. Checked by trace inspection against ground-truth labels.
3. Step count. How many tool and LLM calls did the task take? Two reasons to track: cost (every call is money) and signal (a thirty-step path to a five-step task means the agent is confused). Median and 95th-percentile.
4. Cost per successful task. Dollar cost to complete one successful task. The unit-economics number that must fit your business model.
Without these four on dashboards from day one, the agent is undefended. The single most important question for a vendor is “show me the eval dashboard.” If it is screenshots, you are buying a demo. The companion piece what are AI evals, in plain English covers the broader eval mindset.
A worked example: the contract-clause review agent
Imagine a solo founder building a tool for small-firm lawyers that reviews uploaded contracts (5–50 pages) and flags non-standard clauses against a stored “preferred clauses” library, with a suggested redline.
Signal check. Variable steps — yes, contracts have sections in different orders. External side effects — mild, the system pulls from a library, writes findings, generates a redline. Reasoning between steps — yes, the agent reads a clause, checks it, decides if it is non-standard, and only then generates a redline. Two of three fire; an agent is justified.
The four ingredients, scoped. Planning loop with a twenty-iteration cap and a three-dollar budget per contract. Tool registry of six functions — read_contract_section, get_preferred_clause, compare_clauses, generate_redline, save_finding, finish_review. State object as a JSON document persisted by contract ID. Stopping criterion is the model calling finish_review or hitting a cap.
The eval. Forty real contracts with human-labelled non-standard clauses. Task success — flag every clause the human flagged with at most one false positive. Tool-call accuracy — call get_preferred_clause with the right name. Step count — median fifteen, 95th-percentile twenty-five. Cost per successful task — under one dollar at Claude Sonnet 4.6 pricing.
The cost model. Forty contracts a day per lawyer, $1.50 per contract on Sonnet, $45 per lawyer per month in inference. Pricing at $89 per month leaves defensible margin. A single-call alternative would cost ten cents per contract but achieve sub-fifty-percent task success because the model cannot reliably find clauses scattered across thirty pages in one pass. The agent shape is what makes the product work.
Agentic-washing: 5 vendor red flags
In 2026 every AI vendor calls their product agentic. A short list of red flags that almost always means a workflow or a single call hiding behind the word:
- The proposal does not enumerate the tools. No tool registry, no agent.
- No iteration cap or budget cap is named. The vendor has not modelled runaway cost; you will pay the bill of their first production loop.
- The eval section talks about model quality, not task success rate. The agent is being graded as a single LLM call.
- The architecture diagram is one box labeled “agent.” Without the four ingredients you cannot interrogate the design.
- The cost model is “we’ll see usage in production.” No cost-per-successful-task estimate. The 95th-percentile of agent cost is two to ten times the median; model it before signing.
The decoding “production-ready” in AI agency proposals piece catalogs more procurement-side red flags worth scanning for.
Frequently Asked Questions
What is the simplest definition of an LLM agent?
A language model wrapped in a loop where the model picks which tool to call next, runs it, reads the result, and decides what to do next — until the task is done or it hits a limit. The model decides the order of operations; the engineer does not write it in advance.
Is an agent always better than a single LLM call?
No. Agents are more expensive, slower, and harder to evaluate. They are worth it only when the task has variable steps, requires tool use with side effects, or needs reasoning between steps. For single-shot classification, extraction, or generation, a single call is the right shape.
What is the difference between a workflow and an agent?
A workflow has the sequence of steps written by an engineer in advance — the LLM fills one or more slots. An agent has the LLM choose the sequence at run time, step by step. This is the distinction Anthropic uses in Building effective agents.
How much does an agent cost compared to a single LLM call?
A multi-step tool-using agent costs three to twenty times what a single call to the same model would cost for the same task in 2026. The multiple depends on step count, retries, and tool latency. The 95th-percentile cost is the one that surprises founders in month two.
What is ReAct and why does it keep coming up?
ReAct is the canonical agent-loop pattern from a 2022 paper by Yao and colleagues — reasoning + acting in a loop. Most production agent loops in 2026 descend from ReAct.
Do I need a framework like LangGraph or the OpenAI Agents SDK?
For an MVP, almost never as a starting point. The planning loop is short enough to write directly on the provider’s function-calling API. Frameworks earn their place with multiple agents, complex state, or provider portability.
How do I evaluate whether an agent is working?
Four metrics: task success rate, tool-call accuracy, step count, cost per successful task. All four need dashboards from day one. Without them you cannot tell if a change shipped or regressed.
Can a non-engineer founder spec an agent feature?
Yes — by naming the four ingredients (planning loop, tool registry, state object, stopping criterion), the three justification signals, and the four eval metrics. The vendor or engineer fills in implementation; the founder owns the decision and the audit.
What is the most common mistake founders make with agents in 2026?
Buying an agent when a workflow or single call would do. The agentic premium — in cost, latency, and variance — only earns its keep when at least two of the three need-signals fire.
Closing
An LLM agent is a planning loop, a tool registry, a state object, and a stopping criterion — wrapped around a language model that picks its next move. Above that, everything is marketing. The decision a non-engineer founder owns is not how to build the agent. It is whether to scope one at all, and how to define working once it ships.
The eval-first build playbook covers how to scope any AI feature so it ships at quality. The companion pieces what are AI evals, in plain English and RAG explained for founders cover the two other terms over-sold to non-engineer founders in the same conversations.
One-sentence summary: an LLM agent is an LLM that picks its next move; scope one only when your task has variable steps, real side effects, and reasoning between steps — and only when your business can pay for the eval suite that proves it works.
Dirk Jan van Veen, PhD, is CTO of SFAI Labs, a forward-deployed AI development partner helping non-technical founders ship AI products in 2026.
Dirk Jan van Veen, PhD
