The Eval-First Build Playbook: scoping AI features so they ship at quality

The eval set is the spec. Most AI features fail their quality bar because they were scoped as user stories and then tested as user stories — and AI features do not behave like the features in a 2018 SaaS PRD. They are probabilistic. They drift on every model update. They fail in ways that look fine on the demo and break on the long tail. A user story that says “the agent classifies the email correctly” gives a senior engineer no way to estimate the build, no way to detect a regression, and no way to know when the feature is done. An eval contract — a fixed set of representative inputs, an explicit rubric for what correct means, and a quality threshold — gives them all three. The playbook below is how a founder writes that contract.

This article is the cluster anchor for AI-feature scoping and evals within the broader idea-to-product manifesto. Where the idea validation playbook ends with a defensible PRD, this playbook expands one section of that PRD — the eval contract — into the operational sequence a non-engineer founder uses to scope an AI feature so it ships at quality. The reader is the founder hiring a team. The deliverable is a suite of evals their vendor cannot bait-and-switch on.

Table of Contents

• Why Eval-First Inverts Scoping
• The Nine Stages
  • Stage 1. Capability Inventory
  • Stage 2. Task Taxonomy
  • Stage 3. Rubric Design
  • Stage 4. Test-Set Construction
  • Stage 5. Baseline Run
  • Stage 6. Eval Harness Setup
  • Stage 7. Guardrails and Refusals
  • Stage 8. Regression Suite and CI Gate
  • Stage 9. Handoff Contract
• A Worked Example: A Customer-Support Classifier
• Tool Stack: Promptfoo, Inspect, Langfuse, DeepEval
• The Nine-Question Eval-Readiness Self-Test
• Frequently Asked Questions
• Closing

Why Eval-First Inverts Scoping

A 2018 SaaS feature can be scoped as a user story because the system is deterministic. “When the user clicks Save, the row persists” is testable in a single assertion. The story is the test. A 2026 AI feature cannot be scoped that way because the system is non-deterministic by construction. “When the user clicks Triage, the agent labels the email correctly” is not testable in a single assertion. Correctly is a distribution over thousands of possible inputs, and the right way to specify it is to enumerate a representative sample, write the rubric for what good looks like on each, and set a quality threshold the system must clear.

That enumeration is the spec. The user story without the eval set is a wish. The eval set without the user story is rare — once you have the eval set, the user story usually writes itself.

Three structural reasons make this inversion non-optional in 2026.

First, model behavior changes underneath the product. Frontier models update on cycles of weeks. Anthropic shipped Claude Opus 4 through Opus 4.8 and Sonnet 4.6 over the past year; OpenAI shipped GPT-5 and successor updates; Google shipped Gemini 2.5 family models. Each transition silently changes the behavior of any system pinned to a model alias. A spec without an eval contract is a spec that cannot survive its first model migration (Artificial Analysis LLM Leaderboard).

Second, the long tail is where AI features fail at quality. The demo runs on the inputs the founder hand-picked. The product runs on inputs the founder has never seen. The gap between those two distributions is where roughly 85% of AI pilots stall before reaching production at scale — a figure Gartner has consistently reported through its 2024 and 2025 CIO surveys and that McKinsey corroborates in successive State of AI editions. The eval set, sampled from the actual usage distribution, is the only artifact that closes that gap.

Third, evals are a commercial instrument, not just a testing layer. A founder hiring an agency without an eval contract has no way to grade the work, no way to detect scope drift, and no way to defend the product’s quality bar in front of a customer or investor. The eval set is the asset that turns the agency-founder relationship from a trust exercise into a measurable one. SFAI Labs’ framework on decoding production-ready in AI-agency proposals treats this as the central commercial discipline of an AI build — and the stop scoping AI projects in features thesis is the rhetorical version of the playbook below.

The nine stages that follow turn the eval-first thesis into an operational sequence a non-engineer founder can run. Each stage has a name, an output artifact, a likely tool, the failure mode it prevents, and a “how a non-technical founder reviews it” line. Run them in order. Do not skip the first three because “we already have the spec” — the spec without the eval set is not yet a spec.

The Nine Stages

Stage 1. Capability Inventory

The first stage is to enumerate the discrete capabilities the AI feature actually requires. Most founders, asked to describe their feature, will produce a feature pitch (“the agent triages email and drafts replies and schedules meetings”). A feature pitch is a bundle of capabilities mashed into a single noun. The eval-first playbook starts by unbundling.

The capability inventory lists each atomic thing the model must do, one per row. For an inbox-triage agent: classify the email into one of seven priority buckets; extract the sender’s affiliation; identify whether a reply is requested; detect time-sensitive content; flag promotional or transactional messages; ground the priority decision on the user’s last 30 days of replies; refuse to act on emails containing potential credential resets. Seven capabilities, not one feature. Each one is independently evaluable. Each one fails in a different way.

The inventory is the load-bearing artifact for the rest of the playbook because every later stage operates on capabilities, not on features. The rubric is written per capability. The test set is sampled per capability. The CI gate fires per capability. A founder who skips this stage will end up with a single “is the email triaged correctly?” eval that masks which capability actually broke — and will discover, in week 9, that the priority classifier has been silently regressing for a month because its failures were averaged out against the entity extractor’s stable performance.

• Output artifact: a one-page table listing every atomic capability the feature requires, one row per capability, with a one-sentence description.
• Likely tool: a shared document. No specialized tooling needed.
• Failure mode prevented: aggregated evals that hide which capability regressed.
• How a non-technical founder reviews it: ask “can I describe each row in one sentence without using the word and?” If a row needs an and, it is two capabilities. Split it.

Stage 2. Task Taxonomy

Once the capabilities are inventoried, the next stage is to taxonomize each capability into a task type — because the task type determines the eval design.

The four task types that cover the vast majority of AI feature work in 2026 are:

1. Classification (the model picks one or more labels from a fixed set). Evaluated against ground-truth labels with accuracy, precision, recall, and per-class F1.
2. Extraction (the model pulls structured fields from unstructured input). Evaluated against ground-truth fields with exact-match or fuzzy-match per field.
3. Generation (the model produces free text — a reply, a summary, an explanation). Evaluated with a rubric-graded LLM-as-judge, optionally cross-checked by a human spot-check sample.
4. Tool use / agentic (the model decides whether to call a tool, what arguments to pass, and whether to act on the result). Evaluated against a fixed trajectory: did the model call the right tool, with the right arguments, in the right order?

Mixed-mode features exist — a triage agent does classification plus extraction plus refusals — but the taxonomy lets the team write the right eval per row. The classification row gets accuracy and per-class F1. The generation row gets an LLM-as-judge rubric. The tool-use row gets trajectory grading. Mixing rubrics across task types is the second most common reason eval suites quietly under-detect regressions.

• Output artifact: capability table extended with a task-type column (classification / extraction / generation / tool-use) and a brief note on what correct means for that row.
• Likely tool: the same shared document. Helpful to reference the metric catalogs in DeepEval’s docs for what task-specific metrics look like.
• Failure mode prevented: applying a generation-style LLM-as-judge to a classification task and ending up with a 1–5 score that cannot distinguish a 70%-accurate classifier from a 95%-accurate one.
• How a non-technical founder reviews it: ask “if a customer complained about this row, what would I look at to decide whether they were right?” If the answer is “a label vs. a label”, it’s classification. If it’s “a field vs. a field”, it’s extraction. If it’s “did the model write something good?”, it’s generation. If it’s “did the model do the right thing?”, it’s tool-use.

Stage 3. Rubric Design

The rubric is the load-bearing artifact of the eval suite. It is the document that says, in writing, what good means for each capability. Most eval suites fail not because the harness is broken but because the rubric is fuzzy — a 1–5 score with no anchors collapses to “did I like the answer?”, which collapses to “did the LLM judging the LLM share my taste today?”, which is not a quality bar.

A good rubric has four properties:

1. Anchored. Each score level (1, 2, 3, 4, 5 — or pass/partial/fail) has a written description with at least one example. A 5 is not “great” — a 5 is “all three required fields present, no hallucinated content, response in the user’s locale”.
2. Decomposed. Long rubrics that try to grade everything in one score conceal what failed. The rubric for a generation task should have 3–6 sub-criteria (groundedness, completeness, refusal-correctness, locale-correctness, tone) each scored separately, then combined.
3. Calibrated. Before the rubric is wired into an automated judge, the founder and at least one engineer should grade 20–30 outputs by hand against the rubric and reconcile disagreements. Disagreement on hand-grading is a signal the rubric is ambiguous; tighten the language.
4. Reviewable by a non-engineer. The rubric should read as prose, not as code. If the founder cannot read the rubric and predict how a sample output would be scored, the rubric is engineer-only — which means it is not the founder’s quality bar.

A minimal YAML rubric for a generation capability looks like:

capability: draft_reply
sub_criteria:
  groundedness:
    pass: "Every claim is supported by the input thread or the user's last 30 days of replies."
    fail: "Includes any claim not supported by the inputs."
  completeness:
    pass: "Reply addresses every question or action item in the input email."
    fail: "Omits at least one question or action item."
  locale_correctness:
    pass: "Reply is in the same language as the input email."
    fail: "Reply is in a different language."
  refusal_correctness:
    pass: "If the input requires a credential reset or financial action, reply is a polite refusal with an escalation pointer."
    fail: "Acts on a credential reset or financial action."
aggregation: "All four sub-criteria must pass."

That rubric is short, anchored, decomposed, and a founder can read it. It is also automatable — an LLM-as-judge can score each sub-criterion independently and the harness aggregates.

• Output artifact: a rubric document, one section per capability, with score anchors and at least one example per anchor.
• Likely tool: a shared doc, plus YAML rubric files that the harness (Promptfoo, DeepEval, Inspect AI) can ingest.
• Failure mode prevented: vague rubrics that collapse to “did I like the answer”, producing scores that drift with the judging model.
• How a non-technical founder reviews it: read each sub-criterion aloud and ask “could I, the founder, grade an output against this without asking the engineer what it means?” If not, tighten the language until you can.

Stage 4. Test-Set Construction

The test set is the fixed sample of inputs the eval suite runs against. Three properties matter: size, representativeness, and provenance.

Size. For a single-step classification or extraction capability, 80–150 inputs is the working range. Below 50, the suite cannot distinguish a regression from a flake. Above 300, the team is over-investing in test cases before the product is built. For a multi-step agentic capability with tool calls, the per-capability count is lower (30–60 trajectories) because each one is more expensive to grade, but the total suite size grows with the number of capabilities.

Representativeness. The test set must be sampled from the actual usage distribution, not the founder’s mental model of usage. Pull from the user-conversation transcripts produced in idea validation. Pull from any pilot logs or beta usage. If neither exists, hand-construct inputs based on the user research, but flag those as synthetic and replace them with real inputs as soon as they’re available. A test set sampled from synthetic inputs will pass on the demo and fail on the launch.

Provenance. Every input in the test set has a source — a real customer email, a synthetic case constructed in week 1 of validation, an edge case discovered during a customer call. Tag each input with its source. When the suite regresses, the engineer needs to know whether the failing input is a real production-shaped case or a synthetic one — they’re not equally diagnostic.

A typical test set for a customer-support classifier looks like 120 inputs: 80 sampled from anonymized customer-support tickets, 20 synthetic edge cases (refusal-eligible, multilingual, multi-question), 10 deliberately ambiguous inputs (to validate the refusal-or-escalation path), 10 known-hard cases (where prior agents have failed in production). Each input is tagged with capability, expected label or expected rubric outcome, source, and date added.

• Output artifact: a versioned test-set file (CSV, JSONL, or YAML) with input, expected behavior, capability tag, source tag, and date.
• Likely tool: spreadsheet for authoring; converted to JSONL or YAML for the harness. Langfuse and Promptfoo both ingest these formats.
• Failure mode prevented: a test set sampled from the founder’s mental model that passes on demo day and fails on real inputs.
• How a non-technical founder reviews it: pick five random rows from the test set and ask “does this look like a real user input?” If three or more feel synthetic, the test set is not yet representative.

Stage 5. Baseline Run

Before any prompt engineering, fine-tuning, or model selection, run the test set against the simplest possible system — typically a single prompt against a current frontier model — and record the result. That number is the baseline. Every subsequent change is graded against it.

The baseline does three things. First, it tells the team whether the capability is achievable at all on the current model. A baseline of around 0.40 on a binary classification when the rubric demands 0.85 is a feasibility signal that no amount of prompt engineering will close that gap; the team should consider scoping down, fine-tuning, or pausing the project. Second, the baseline anchors the budget. If a customer-support classifier hits, hypothetically, around 0.78 on a 100-prompt test set with a single off-the-shelf prompt, then the build budget should be sized for the work of going from 0.78 to 0.92, not for the work of going from scratch to 0.92. Third, the baseline reveals failure modes. Read the 22 failing cases. Group them. Five of them might be in one language the prompt didn’t account for; eight might be a single edge case the prompt mishandles; three might be genuinely ambiguous and belong in the refusal bucket, not the classification bucket. Failure-mode analysis is where the rubric and the test set get tightened before the build starts.

Whatever number the baseline produces, cite it as the baseline — not as a benchmark. Benchmark numbers belong to public reports (Anthropic’s system cards, OpenAI’s tech reports, the Artificial Analysis leaderboard, arXiv submissions). A team’s own baseline is a measurement of a specific test set against a specific model with a specific prompt. It is the team’s anchor; it is not a benchmark.

• Output artifact: a baseline report listing the test-set pass rate per capability, the model and prompt used, and a failure-mode summary.
• Likely tool: Promptfoo for the run; spreadsheet or markdown for the failure-mode analysis.
• Failure mode prevented: building toward a quality bar without knowing how far away you are from it — and budgeting for the wrong amount of work.
• How a non-technical founder reviews it: ask the engineer to walk you through the failures, not the passes. If the engineer cannot group the failures into 3–5 patterns, the test set is either too noisy or the rubric is fuzzy.

Stage 6. Eval Harness Setup

The harness is the program that runs the test set against the system under test, applies the rubric, aggregates the scores, and writes a report. There are four credible options in 2026:

• Promptfoo — open-source, YAML-config, good for early-stage teams and prompt comparison. Strong CI story.
• Inspect AI — UK AISI’s open-source framework, Python-native, originally built for safety and dangerous-capability evals. Solver/scorer architecture maps well onto agentic features.
• Langfuse — open-core, observability-first. Strong for production-trace evaluation and for teams who want offline and online scoring in one place.
• DeepEval — open-source Python library with a pytest-style API and a metric catalog (G-Eval, RAG metrics, hallucination, faithfulness). Easiest to integrate into existing test infrastructure.

The choice depends on context. A team with a Python-heavy stack and a strong testing culture will be at home with DeepEval. A team running production traces and wanting one tool for offline + online evals will pick Langfuse. A team doing safety-leaning or agentic evals will pick Inspect. A team optimizing fast prompt iteration will pick Promptfoo. None of them is wrong; what matters is that the harness is wired up before the build starts, not bolted on after the demo.

A minimal Promptfoo config for a classification capability looks like:

prompts:
  - file://prompts/classify_email_v1.txt
providers:
  - id: anthropic:claude-sonnet-4.6
  - id: openai:gpt-5
tests:
  - file://test-sets/triage-classifier.jsonl
defaultTest:
  assert:
    - type: equals
      value: "{{expected_label}}"

That config runs the prompt against both providers, scores each output against the expected label, and produces a pass-rate report. The team uses the same config in CI (Stage 8).

• Output artifact: a working harness configured against the test set, with a documented run-evals command that produces a report.
• Likely tool: one of Promptfoo, Inspect AI, Langfuse, DeepEval (linked above).
• Failure mode prevented: ad-hoc evals run in notebooks that nobody else can reproduce; results that depend on which engineer ran them and when.
• How a non-technical founder reviews it: ask “if I were here at midnight and wanted to know whether the system passes its evals, what would I run?” The answer should be one command. If the answer involves opening a notebook and explaining what to skip, the harness is not yet a harness.

Stage 7. Guardrails and Refusals

An eval suite that only tests the happy path is a suite that ships the happy path. Guardrails and refusals are the cases where the right behavior is not to do the thing — refuse a request to reset a credential, refuse to commit to a discount the agent has no authority to grant, escalate an ambiguous case rather than guess, decline to operate on PII without the right scopes.

Every AI feature has a refusal surface. It is rarely as obvious as the happy path, and it is consistently under-tested. The eval suite must include a dedicated refusal-and-guardrail section — typically 10–30 inputs per capability — where the expected behavior is a refusal, an escalation, or a no-op. Grading these inputs as classification (refusal vs. action) is usually cleaner than grading them as generation.

This stage also covers PII handling and prompt-injection resistance. A customer-support classifier should not exfiltrate names or addresses into a summary it sends to a third-party model. An agentic feature should not be steerable by adversarial inputs into ignoring its instructions. The 2024–2026 trend toward agentic systems has made prompt injection one of the higher-risk operational categories — Simon Willison’s prompt-injection writeups and OWASP’s LLM Top 10 are the canonical references.

The guardrail evals do not have to cover every imaginable attack. They have to cover the ones that, if they fired, would produce a customer-facing or PR-facing incident. Sample 10–20 plausible injection attempts, 10–20 PII edge cases, and 10–20 over-broad-permission requests. Wire them into the suite. Re-run on every change.

• Output artifact: a refusal-and-guardrail test sub-suite (10–30 inputs per capability) with expected refusal behavior tagged explicitly.
• Likely tool: the same harness as Stage 6; for prompt-injection coverage, tools like Promptfoo’s red-team module or Patronus provide pre-built adversarial inputs.
• Failure mode prevented: shipping a feature whose first production incident is a refused-action-that-acted-anyway story on social media.
• How a non-technical founder reviews it: ask “what’s the worst thing the feature could do to a customer that the eval suite would catch before launch?” If the answer is fuzzy, the refusal suite is fuzzy.

Stage 8. Regression Suite and CI Gate

The eval suite becomes a regression suite the moment the team commits to running it on every change. CI integration is the operational discipline that turns a one-time eval into a quality contract.

Three patterns are well-established in 2026:

1. PR gate. Every pull request runs the suite. The PR cannot merge if pass rates drop below the agreed threshold on any capability. The harnesses above all expose this as a single command that returns a non-zero exit code on regression.
2. Nightly run on production traces. A sample of the previous 24 hours of real production traffic is replayed against the current system, scored, and reported. This catches drift that the static test set misses — typically caused by user-behavior shifts or upstream tool changes.
3. Model-migration replay. Before any model alias upgrade (Sonnet 4.6 → 4.7, GPT-5 → next), the suite runs against the new model, the report is diff’d against the prior model, and any capability that regressed by more than the agreed delta blocks the migration.

The CI gate is also the line between “evals exist” and “evals matter”. A team that runs evals but does not gate merges on them will gradually let the suite rot — engineers will skip flaky cases, ignore drift, and the suite will fall out of sync with the system it tests. The gate is what keeps the suite alive.

• Output artifact: a CI workflow (GitHub Actions YAML or equivalent) that runs the eval suite on every PR and reports pass rates per capability, plus a nightly job replaying production traces.
• Likely tool: GitHub Actions / GitLab CI, plus the Stage 6 harness; for production replay, Langfuse’s trace replay or a custom job built on the harness.
• Failure mode prevented: the suite that quietly stops running, then quietly stops passing, then is quietly deleted six months later because “the evals don’t reflect reality anymore”. The CI gate is the cheapest insurance against that exact decay.
• How a non-technical founder reviews it: ask the engineer to show you a PR where the suite blocked a merge. If they cannot find one — and the team has been shipping for more than a month — the gate is not really gating.

Stage 9. Handoff Contract

The final stage is to wrap the suite into a handoff contract — the document that names the eval suite as the basis on which the agency, internal team, or contractor will be judged.

A handoff contract has six parts:

1. The eval suite itself, versioned and shared.
2. Per-capability quality thresholds: the pass rate below which the feature is not shipped.
3. The CI gate: the suite runs on every PR; the team cannot merge below threshold.
4. The model-migration clause: any model alias change re-runs the suite; the founder is notified of any capability with a delta above the agreed bound.
5. The expansion clause: the founder retains the right to add cases to the suite at any time during the engagement. The agency cannot block additions.
6. The dispute-resolution clause: when the agency and the founder disagree on whether the feature is shipped, the eval report is the deciding artifact. If the suite passes at the agreed thresholds and the founder is still unhappy, the dispute is a rubric dispute, not a build dispute, and the rubric gets refined for the next milestone.

This is what the SFAI Labs writeup on stop paying AI agencies for documentation, pay them for evals is operationalizing. It is also the conversation that prevents almost every scope-drift dispute we have seen on AI builds — once both sides are arguing over the same eval numbers, the discussion is technical, not political.

• Output artifact: a 1–3 page handoff contract, signed by both parties, referencing the live suite.
• Likely tool: a shared doc; the decoding production-ready framework is a useful template for the surrounding language.
• Failure mode prevented: the week-12 dispute where the founder thinks the feature isn’t done, the agency thinks it is, and neither party can point to an objective artifact to settle the question.
• How a non-technical founder reviews it: read it aloud and ask “if my engineer and the agency engineer disagree at week 9, does this document tell us who is right?” If not, the contract is incomplete.

A Worked Example: A Customer-Support Classifier

To make the playbook concrete, run it through a hypothetical SaaS company building a customer-support classifier. The feature: when a new support ticket arrives, classify it into one of seven buckets (billing, authentication, feature request, bug report, integration, refund request, abuse), extract the customer’s stated urgency, and detect whether the ticket requires a refusal (credential resets, payment-action requests).

Stage 1 capability inventory produces seven rows: classify into seven buckets; extract stated urgency on a three-point scale; detect refusal-eligible content; detect language; detect product-version mention; flag legal or regulatory keywords; route to the right team. Seven capabilities, four task types.

Stage 2 taxonomy assigns: classification (rows 1, 4, 7), extraction (rows 2, 5, 6), guardrail-classification (row 3). Each row gets its own metric — accuracy and per-class F1 for the classification rows, exact-match and fuzzy-match per field for the extraction rows, and a separate refusal-precision metric for the guardrail row.

Stage 3 rubric design writes anchors for each capability. For the seven-bucket classification: a 1-class label is correct, multi-label tickets get the dominant label, the rubric explicitly resolves the billing-vs-refund ambiguity in favor of refund when the customer asks for money back. The founder reads the rubric, predicts how a sample ticket would be scored, and refines the language where their prediction diverges from the engineer’s.

Stage 4 test-set construction pulls 100 anonymized historical tickets from the company’s last 90 days of support data, plus 20 synthetic edge cases (multilingual, refusal-eligible, ambiguous between two buckets), plus 10 known-hard cases the previous rules-based classifier failed on. Total: 130 inputs, each tagged with capability, expected label, source, and date.

Stage 5 baseline run executes a single prompt against Claude Sonnet 4.6 on the full 130. The hypothetical baseline report: 92 of 130 pass (about 0.71), with failures concentrated in the multilingual subset (the prompt didn’t account for Spanish tickets) and in the billing-vs-refund ambiguity (the prompt’s instructions for the tie-breaker were imprecise). The founder reads the failures, refines the rubric on the billing-vs-refund tie-breaker, and the engineer adds two lines of Spanish handling to the prompt. The next run hits, hypothetically, around 0.82. The team agrees the per-capability threshold for shipping is 0.85 on classification, 0.90 on extraction, and 0.95 on refusal (refusals are higher because the asymmetric cost of a false negative — actioning a credential reset — is severe).

Stage 6 harness setup wires the suite into Promptfoo. A single npx promptfoo eval command produces the per-capability pass-rate report. The team runs it locally on every prompt change.

Stage 7 guardrails and refusals adds 25 refusal-eligible inputs (credential resets, payment actions, abuse threats) and 15 plausible prompt-injection attempts. The team grades these as a separate sub-suite.

Stage 8 regression suite and CI gate integrates the suite into GitHub Actions. Every PR runs the full 170-input suite (130 main + 40 refusal). A drop below threshold on any capability blocks the merge. A nightly job replays the previous 24 hours of real tickets through Langfuse.

Stage 9 handoff contract names the suite, the thresholds, the gate, the model-migration replay, and the expansion clause. The agency and the founder sign it. The build begins.

The founder, who is not technical, has not written a line of code. They have written a rubric, reviewed a test set, read a baseline failure report, and signed a handoff contract. They have scoped an AI feature so it ships at quality.

Tool Stack: Promptfoo, Inspect, Langfuse, DeepEval

The four tools named through the playbook are not interchangeable, but they overlap. A short positioning matrix:

Tool	Best for	Stack	Weakness
Promptfoo	Fast prompt iteration, CI integration, red-team coverage	YAML config, Node/TS-native	Less strong on production observability
Inspect AI	Agentic and safety-leaning evals; solver/scorer composition	Python-native, AISI-backed	Heavier setup; safety-research idioms
Langfuse	Production trace evaluation, offline + online in one tool	Open-core, multi-language SDKs	Observability-first; less rubric-design help
DeepEval	Existing Python test culture; rich metric catalog	Python-native, pytest-style	Library-shaped; less of a full platform

In practice many teams run two: Promptfoo or DeepEval for offline / CI evals, Langfuse for production traces. Inspect shows up most often in teams with a safety-research or agentic emphasis. None of these tools writes the rubric for you — that is the work this playbook is about.

The hosted commercial alternatives (Braintrust, Patronus, Arize Phoenix) offer additional capabilities — managed UIs, more sophisticated experiment tracking, pre-built adversarial suites — but the open-source four cover the vast majority of feature-scoping work for a non-engineer founder hiring a team.

The Nine-Question Eval-Readiness Self-Test

Before commissioning a build, the founder should be able to answer yes to all nine:

1. Have I enumerated every atomic capability the feature requires, one per row, no and?
2. Have I assigned a task type (classification, extraction, generation, tool-use) to each capability?
3. Have I written a rubric for each capability with anchored sub-criteria a non-engineer can read?
4. Have I assembled a test set of 80–150 representative inputs per capability, with provenance tagged?
5. Have I run a baseline on a current frontier model and read the failure modes?
6. Have I picked an eval harness (Promptfoo, Inspect, Langfuse, DeepEval) and wired the suite into a single command?
7. Have I added a refusal-and-guardrail sub-suite of 10–30 inputs per capability, plus prompt-injection coverage?
8. Have I gated CI on the suite so PRs cannot merge below threshold, and scheduled a nightly production-trace replay?
9. Have I signed a handoff contract with the team that names the suite as the basis of judgment?

Nine yeses means the feature is scoped in evals. Fewer than nine means the team is about to budget for a build that may or may not ship at quality — and the founder will discover which by the week-9 dispute.

Frequently Asked Questions

What is eval-first development in plain English?

Eval-first development is the practice of writing the test set for an AI feature before writing the feature. The test set — a fixed sample of representative inputs plus a written rubric for what the right output looks like on each — becomes the spec. The build is then graded against it on every change. It inverts the 2018 SaaS pattern, which wrote the spec first and the tests later. For AI features, the spec without the eval set is a wish; the eval set is the spec.

Are evals the same as benchmarks?

No. Benchmarks (MMLU, GPQA, SWE-Bench Verified, HumanEval) are public test sets used to compare models across the industry; they live on the Artificial Analysis leaderboard and in Anthropic and OpenAI system cards. Evals are your team’s private test sets that measure your feature on your distribution. Benchmarks tell you which model is generally strong; evals tell you whether your feature ships at quality. Both matter, for different decisions.

How big should the eval suite be at scoping time?

For a single-capability feature, 80–150 inputs is the working range. For a multi-capability feature, the per-capability count holds (80–150 each), with smaller counts (30–60) for expensive-to-grade agentic capabilities. Refusal sub-suites add 10–30 inputs per capability. A scoping-time suite total of 150–500 inputs is normal for a small to mid-sized AI feature.

Who writes the rubric — the founder or the vendor?

The founder writes it in prose; the vendor formats it for the harness. The rubric is a domain artifact — what good means for your users is your knowledge, not the vendor’s. A vendor who writes the rubric without your input is writing a rubric the model can pass, not a rubric that protects your users. Sign off on every line.

Can the eval suite be written entirely by an LLM?

No, not for scoping-time work. An LLM can help draft synthetic test inputs and propose rubric language, but it cannot decide what correct means for your feature, cannot tell you which inputs are representative of your usage distribution, and cannot calibrate the rubric against your customers’ expectations. The founder owns the rubric. The LLM is a drafting assistant.

How does the eval suite handle model updates?

The suite is the migration test. Before any model alias upgrade — Sonnet 4.6 to a successor, GPT-5 to a next version — the suite runs against the new model, the report is diff’d against the prior model, and any capability that regresses by more than the agreed delta blocks the migration. The handoff contract names this clause explicitly. Teams without this clause discover regressions in production, not in CI.

What if my AI feature is genuinely creative — drafting marketing copy, writing summaries — and the rubric feels subjective?

Then decompose the rubric harder. Even creative tasks have measurable sub-criteria: groundedness on the brief, locale-correctness, refusal-correctness on out-of-scope requests, length-correctness, tone-match against examples. The LLM-as-judge grades each sub-criterion against anchored language, with human spot-checks on a 20-output sample per cycle to keep the judge calibrated. Subjective usually means insufficiently decomposed — push the decomposition until each sub-criterion has a defensible anchor.

Why does this matter to a non-technical founder?

Because the eval suite is the artifact that determines whether you get the product you paid for. Without it, you are paying the agency or the internal team for output you cannot grade, and the disputes that follow are scope disputes — political and unwinnable. With it, the disputes are technical: the suite passed or it didn’t, the rubric is right or it isn’t, the threshold is correct or it should move. Technical disputes are resolvable. The founder who scopes in evals is the founder who keeps the upper hand through the build.

Can I run this playbook for under $10K of consulting cost?

Most non-engineer founders can run stages 1–4 (capability inventory, taxonomy, rubric, test set) themselves with 8–12 hours of pairing with a senior engineer at typical contractor rates. Stages 5–8 (baseline, harness, guardrails, CI gate) are engineer-heavy and benefit from a few days of an evaluation engineer. Stage 9 (handoff contract) is back to the founder. A total scoping engagement is typically a 1–2 week effort and a moderate consulting spend — small in the context of a full build.

What is the relationship between evals and the PRD?

The PRD is the document the engineer estimates against; the eval suite is the artifact the team is graded on. In a well-scoped 2026 idea-to-product engagement, the PRD references the eval suite as its acceptance criteria. The idea validation playbook ends with a PRD whose stage-4 section is the eval contract. This playbook unfolds that section into the operational sequence.

Closing

The eval-first build playbook is not a quality assurance afterthought. It is the scoping framework that determines whether an AI feature ships at the quality the founder signed for. Each of the nine stages — capability inventory, task taxonomy, rubric design, test-set construction, baseline, harness, guardrails, CI gate, handoff contract — is a check the playbook makes mandatory because the alternative is paying for a build that may or may not work and discovering which during a week-9 dispute that no contract can settle.

A non-engineer founder who runs this playbook does not become an engineer. They become a buyer who can grade what they bought. That is the control surface the rest of the idea-to-product manifesto depends on. The sibling cluster anchors — the AI MVP economics playbook, the DIY with AI manifesto, the founder-AI partner operating manual — each assume the eval contract is in place. Without it, every downstream decision (budget, build/buy, cadence) is being made against a quality bar nobody has written down.

Write the eval set. Then write the feature. Then ship.