Most AI MVP-1s should ship with a single, well-structured prompt — no retrieval, no fine-tuning, no chains. The instinct in 2026 is the opposite. A founder hears “AI product” and pictures a stack — vector store, reranker, fine-tuned base, orchestrator. The agency proposal arrives with all four line items, the build calendar swells to 14 weeks, and the engineering bar for v1 quietly moves from ship a feature users come back to toward ship the architecture the deck promised investors. The cheap correction is unfashionable: scope v1 around what a single prompt against a frontier model can do, ship that, and let real-user data say whether you need more. Around seven in ten AI MVP-1s pass this test; the rest fail loudly enough that the migration path is obvious by month three.
This piece extends the Eval-First Build Playbook, part of the Idea-to-Product Manifesto. The playbook argues you scope AI features in evals; here we apply that lens to technique choice. Related reading: the case for boring AI features in MVP-1 and what is fine-tuning vs prompting vs RAG. The polemic companion is stop scoping AI projects in features — scope them in evaluations.
Decision Scope
A scoping POV for AI product MVP-1s. The 70%+ rate is a planning heuristic drawn from agency proposal patterns and the customization-ladder guidance Anthropic, OpenAI, and Google publish in 2025–2026 — not a peer-reviewed statistic. Strong default, not a law. Dollar and percentage figures in worked examples are illustrative.
What “prompt-only” actually means in 2026
Five properties define a prompt-only AI feature. Any deviation and it isn’t prompt-only.
- One LLM call per request. No chains, no agent loops. One synchronous round-trip from input to structured output.
- A single, well-structured prompt. Persona, task, rubric, few-shot examples, output schema. Long, deliberate, version-controlled — not the three-liner a founder types into ChatGPT.
- No external retrieval. No vector store, no semantic search. Any context lives in the prompt or arrives in the request payload from your own database.
- No model training. No fine-tuning, no LoRA. Base model as the vendor ships it.
- A frontier-tier model. In 2026: Claude Opus 4.8 or Sonnet 4.6, GPT-5, or Gemini 2.5 for the upper end; Claude Haiku 4.5, GPT-5 mini, or Llama 3.3 for high-volume. Prompt-only on a weaker model is false economy — the gap shows up immediately in evals.
Most “but we’re prompt-only” claims fail at property 2. A founder shows me a prompt living in a .env file as a 30-word instruction nobody owns. That isn’t a prompt — it’s a comment. A prompt-only MVP needs an actual prompt: 300–1,500 tokens, sectioned, reviewed like a contract, graded against evals before shipping. Prompt-only without an eval set is wishful thinking with fewer dependencies. The eval-first build playbook is the prerequisite.
Why founders skip past prompt-only
Three reasons recur, all the same shape: prompt-only sounds less impressive, so founders skip it before testing whether they need to.
The agency proposed RAG. RAG sounds like a product — vector DB, reranker, chunking, embedding model. Four people, 8–12 weeks. Prompt-only is harder to bill; no architecture diagram for the slide.
The investor saw a stack diagram. Eight boxes in the deck — model, retrieval, evals, tools, guardrails, memory, agent, observability. By kickoff, the founder feels the diagram is a commitment. Architecture comes first, feature second.
The founder believes complexity is differentiation. An identity claim. The thesis is that the stack is the moat. The argument has a kernel of truth — a raw prompt is copyable in five minutes — and gets the conclusion wrong. Defensibility comes from the data flywheel a prompt-only feature collects on real users, not from the architecture under it. Skipping prompt-only doesn’t preserve a moat; it delays getting one.
The compounding cost is real. Prompt-only v1 ships in 2–4 weeks; the same feature with retrieval, 8–12. The 6–8 week delta is lost user data, lost iteration, lost cash burn. By the time the RAG-first MVP hits a user, prompt-only would already have been graded against three rounds of real-user failures — the very information that would have told you whether you needed RAG.
Five capabilities prompt-only does well in 2026
The 2025–2026 frontier expanded what a single prompt can deliver. Five categories now sit clearly inside the prompt-only envelope. If your MVP-1 feature is one of them, prompt-only is the default — not a fallback.
1. Classification and extraction. Return a category label, a set of fields, or a routing decision. Frontier models perform within a small margin of fine-tuned classifiers on most generic-domain tasks. Ships prompt-only: ticket triage, lead enrichment, contract-clause classification, narrow-surface intent detection.
2. Summarization and rewriting. Long input → shorter or reformatted output. Brand-voice rewriting, meeting-note compression, multi-document synthesis (when the docs fit), customer drafts from internal notes. A 600-word prompt with three few-shot examples gets most of the way to a fine-tuned style model — and the few-shot block updates in five minutes when style drifts. The fine-tune cannot.
3. Structured Q&A over short context. When supporting context fits in the prompt, retrieval is overhead. Gemini 2.5 ships a 1M+ token window; Claude and GPT-5 have aggressive long-context modes. Prompt-only Q&A over a single document, profile, or ticket history beats a poorly tuned RAG pipeline. The threshold isn’t “is there context?” — it’s “is the context bigger than the model’s effective working window for this task?” In 2026 that threshold is materially higher than in 2024.
4. Tool-routing and dispatch. Read input, pick the downstream function, produce structured arguments. The model returns a JSON object naming tool and arguments. Frontier models do this reliably with a tight prompt. Prompt-only tool-routing is one call; multi-step agent loops are not prompt-only and belong in MVP-2 unless the eval contract demands them on day one.
5. Reformatting and enforcement. Extract into a JSON schema, rewrite to fit a format, validate against constraints. Strict JSON modes and schema-constrained decoding make this boringly reliable. Retrieval or fine-tuning is never the right answer to reformatting; if the agency proposed it for a reformat task, that’s a smell.
The five share a shape: bounded input, bounded output, an eval rubric writable in a day, no need for the model to read anything beyond the prompt. That’s an MVP-1 AI feature.
Three signals you actually need more
Prompt-only by default, not prompt-only forever. Three signals should change the default — only these three.
Signal 1: The eval fails on the strongest frontier model with the best prompt you can write. A 1,200-token structured prompt, run against the strongest available model (Opus 4.8, GPT-5, or Gemini 2.5 — whichever scores best on your category), iterated three times against failures. The eval still doesn’t clear threshold. The missing ingredient is information or behavior the model doesn’t have. Missing information → retrieval. Missing behavior → fine-tuning. Most signal-one failures point to retrieval, because the missing ingredient is usually the user’s private documents.
Signal 2: Context overflow past the model’s effective working window. Not the marketing context window (those are huge), but the effective one — frontier models in 2026 still degrade at retrieval-style tasks deep into long contexts. When measured recall starts dropping at 200K, 400K, or 800K tokens depending on the task, you’ve hit the prompt-only ceiling. This almost always means RAG. The exception: structured context (DB rows, ticket histories) where query-shaped retrieval from your own system beats a vector store.
Signal 3: Hard-required style or format adherence prompting cannot reach. The output must follow a specification so narrow that few-shot drifts off-spec at a rate that breaks the use case — legal contracts in a firm’s house style, medical-record formatting under a strict template. You can demonstrate the failure rate against a held-out set, show that adding examples doesn’t close the gap, and show the gap matters to the user. The next step is fine-tuning a small open-weights model (Llama 3.3 in 2026 is the common landing spot) on a few thousand high-quality examples. This is the rarest signal and the most expensive to act on. If you can’t write the failure rate down as a number from a measured eval, you have an aesthetic preference, not signal three.
What’s not a signal: “the demo felt slow,” “an investor asked about our stack,” “a competitor announced a fine-tune.” Move only on the three above.
The migration path
When a signal fires, migrate one rung at a time. Skipping rungs costs build time and obscures which addition actually moved the eval.
- Prompt-only (MVP-1 default). Single LLM call, frontier model, structured prompt with eval.
- Prompt-chained. Two or three sequential calls — draft, critique, finalize — without retrieval or training. Often the cheapest fix for signal-one failures.
- Tool-augmented. The call invokes tools (DB query, API, calculator). Reliability on factual subtasks without retrieval infrastructure.
- Retrieval-augmented (RAG). Vector store, embedding model, retrieval ahead of generation. The right answer to signal two and many signal-one failures.
- Fine-tuned. A custom-trained small open-weights model for narrow style or domain adherence. The answer to signal three; rarely one or two.
A common error in proposals is skipping rungs 2 and 3 because RAG sounds like the “real” architecture. Rungs 2 and 3 are where most signal-one failures resolve, and they’re cheap. Make the agency demonstrate prompt-chaining and tool-augmentation failed before accepting RAG scope.
Worked example: a research-summarizer that shipped prompt-only
Illustrative — a representative shape, not a deployment claim.
A B2B SaaS founder wants a research summarizer. The user pastes a URL; the product returns a 300-word summary in the company’s house tone plus three implications tagged to source sections.
Agency proposal. 10 weeks: scrape, chunk, embed, retrieve, pass to a summarization model fine-tuned on the company’s prior reports. Four engineers. Roughly $180K, illustratively.
Prompt-only counter. Four weeks. One frontier-model call per request. Page text (up to ~200K tokens) drops into a prompt with persona, rubric, five-shot block, and output schema. One engineer. Roughly $40K, illustratively.
Eval contract. Twenty hand-graded inputs (ten papers, five reports, five blog posts). Threshold: 17/20 ship-ready, zero hallucinations.
v1. Shipped in three weeks; cleared 18/20 first run. Two failures on very long analyst PDFs. Founder shipped.
Month one. 1,200 summaries by 80 paying users. Five complained implications cited content from older reports pasted in previous sessions. Three asked whether the product could synthesize across multiple documents in one summary.
Migration. Cross-document synthesis is signal two (overflow) plus signal one (a new feature, not a regression). v2: six weeks, per-user document store, vector retriever, retrieval from the user’s own corpus at summary time. RAG, on evidence — real-user signals, measured failure modes, an extension to the eval contract. Two engineers. Roughly $80K, illustratively.
Had the original $180K, 10-week proposal landed, v1 ships in week 10 and cross-document in week 16. The founder would have spent $180K and 10 weeks to learn what $40K and 3 weeks of prompt-only surfaced. Architecture follows data, not the deck.
The prompt-only scoping rubric
Walk these for each AI feature in MVP-1 before accepting any technique above rung 1.
| Question | If “no” | If “yes” |
|---|---|---|
| Eval rubric writable in one day? | Re-scope; see boring AI features in MVP-1. | Continue. |
| Context fits comfortably in 1M tokens for v1? | Likely signal two; RAG may be warranted. | Continue. |
| Task is one of: classification, summarization, short-context Q&A, tool-routing, reformatting? | Audit which non-prompt-only rung the task needs; demand the agency justify it. | Continue. |
| Wrote a 600–1,500-token structured prompt and ran it against the eval set on a frontier model? | Do this first. The default isn’t “skip prompt-only” — it’s “do prompt-only properly first.” | Continue. |
| Prompt-only run cleared the threshold? | Move up exactly one rung, re-run, measure. | Ship. Re-evaluate at month one. |
If a proposal includes anything above rung 1 for a feature that passes this rubric, ask the agency to justify the rung in writing. Rung-1 cost is the starting point; rung 2+ gets added only when an eval forces it.
Frequently asked questions
Is prompt-only the same as “just use ChatGPT”? No. “Just use ChatGPT” is a person typing into a chat window. Prompt-only is a version-controlled, eval-graded prompt deployed against an API as part of a product. Same model, very different artifact.
What if my product needs to know my company’s documents? Check whether the documents fit in the model’s working context window when included in the prompt. In 2026, frontier models handle 200K–1M tokens cleanly on many task types. If they fit, prompt-only with the documents included is the answer. If they don’t, you’ve hit signal two.
Doesn’t prompt-only make my product easy to copy? The prompt is copyable in five minutes; the eval set, the labeled failure data from your users, and the iteration history that produced the prompt are not. The defensible asset of an AI product is the data flywheel from real usage, not the architecture under it. Prompt-only ships the flywheel faster.
What models should I use for prompt-only in 2026? For strong judgment: Claude Opus 4.8, Sonnet 4.6, or GPT-5. For high-volume cost-optimized: Claude Haiku 4.5 or GPT-5 mini. For long-context Q&A: Gemini 2.5. For self-hosted: Llama 3.3. Test on at least two model families.
How do I know my prompt is well-structured and not just long?
Named sections: persona, task, rubric, few-shot examples, output schema, failure-mode handling. Reviewed like a contract. Versioned in git. Tested against the eval set on every change. A paragraph in a .env file is a placeholder, not a prompt.
Isn’t a frontier model expensive per call? Per call, yes — but the per-call cost is almost always a small fraction of the all-in cost of operating a RAG pipeline (vector DB, embedding model, retrieval ops, dual-layer evals). At MVP-1 volumes, prompt-only on a frontier model is cheaper end-to-end in the large majority of cases.
When is fine-tuning the right MVP-1 call? Almost never. Fine-tuning fits when (a) the task requires narrow style or format adherence prompting demonstrably cannot reach, (b) you already have a few thousand high-quality examples, and (c) volume is high enough that inference savings exceed training and ops cost. If you can’t tick all three, default to prompt-only.
What if my agency only quotes RAG? Ask for a parallel quote for a prompt-only v1 with a clear migration path to RAG on signal two. A competent agency in 2026 produces that quote in 48 hours. If they can’t, see the field guide to evaluating an AI agency in under 90 minutes.
How does prompt-only interact with eval-first? Eval-first is the method; prompt-only is the default technique under that method. The eval-first build playbook scopes AI features in evals; this piece names the technique that clears them in MVP-1.
Where do I go next? Download the AI MVP scoping worksheet — the rubric above as a fillable doc you bring to your agency review.
Dirk Jan van Veen, PhD
