Extraction accuracy is the share of fields a document AI system reads correctly when it pulls structured values out of a document. If a tool extracts the tenant name, base rent, and renewal date from a lease and gets the rent wrong, that is one miss against however many fields it attempted. A “95% accuracy” claim means that, on some test set, roughly 95 of every 100 field values came out matching a correct answer. That sentence sounds precise, and it hides almost everything a small commercial real estate firm needs to know: 95% of which fields, graded against whose answer key, and 95% per field or per document. This piece defines the term from the ground up, shows how the number is actually measured, and works out what 95% turns into on a real lease so you can read the figure without a data scientist in the room.
What extraction accuracy actually measures
Extraction is the step where software turns a document into data. A lease is prose and tables; a lease abstract is a row of named values — tenant, premises, commencement date, base rent, escalation, renewal option, notice window. Extraction accuracy scores how many of those named values the system got right. If you have never built one of these rows by hand, our explainer on what lease abstraction is walks through the fields a good abstract captures; extraction accuracy is simply the grade on that row.
The unit of measurement matters. Accuracy is almost always reported at the field level: each extracted value is compared to a known-correct value, scored right or wrong, and the results are averaged. So “95%” usually means that across all the fields in all the test documents, 95% of individual values matched. It does not, on its own, tell you how many whole documents came out clean, which is a different and much lower number. Holding those two ideas apart is the single most useful thing a buyer can learn about this metric.
Reading is not the same as extracting
The first place a “95%” claim can mislead is by quoting the wrong task. Two things happen when AI processes a scanned lease, and they have separate accuracy scores.
- Reading (OCR). Optical character recognition converts the pixels of a scanned page into text characters. Its accuracy answers: did the system turn the image into the right letters and numbers? On a clean, digitally native PDF this is close to a solved problem.
- Extracting. Extraction decides what each value is: that this date is the renewal notice deadline and not the default cure period, that this number is the base rent and not the security deposit. This is the harder task, and the one that determines whether your abstract is correct.
A vendor can honestly report 99% character accuracy on reading while extraction on the clauses you care about is far less reliable — the two get quietly swapped in a pitch because the reading number is higher and easier to hit. When you see an accuracy figure, the first thing to pin down is which task it describes. The number that governs whether your data is trustworthy is extraction accuracy, not read accuracy.
How accuracy is measured: the answer key
Here is the part almost no vendor page shows you, and it demystifies the whole metric. Accuracy is measured against an answer key that a human built.
The process is the same one a teacher uses to grade a test:
- Take a set of documents — ideally real leases with real amendments.
- Have a person read each one and record the correct value for every field. This is the ground truth, the answer key.
- Run the AI system on the same documents.
- Compare the system’s output to the answer key, field by field, and count matches.
Accuracy is matches divided by attempts. That is the entire mechanic. Which means two things follow immediately. First, the score is only as honest as the answer key: if the person who built ground truth misread a clause, the AI gets marked wrong for being right, or vice versa. Second, the score depends on what counts as a “match,” and there is real choice there:
- Exact match requires the value to equal the answer character for character, so “$4,500” and “4500” are scored as different.
- Normalized match cleans up formatting first, so “NNN” and “triple net” count as the same, which is closer to how a person would grade.
- Human-judged match lets a reviewer decide whether the extraction is usable — most realistic, least reproducible, and the most flattering if the grader is lenient.
None of these is wrong, but they produce different percentages from the identical output. A tool reporting 92% on strict exact match may be more reliable than one reporting 97% on lenient human grading. When a claim does not name its answer key or its match rule, the number is not yet decodable — a point we push further in our guide to reading a vendor’s accuracy claim.
What 95% means for one lease
Now the arithmetic that reframes the number, and that no marketing page runs for you. Field-level accuracy and document-level accuracy are not the same figure, and the gap between them is wide.
Suppose a tool is genuinely 95% accurate per field. A working lease abstract has roughly 30 fields. The chance that a single document comes out with every field correct is about 0.95 multiplied by itself 30 times — 0.95 to the 30th power — which is around 21%. A per-field number that sounds excellent means that, left unchecked, close to four out of five documents contain at least one error somewhere.
The sensitivity to that per-field figure is worth internalizing, because it is not linear:
| Per-field accuracy | 30-field document fully correct |
|---|---|
| 95% | ~21% |
| 97% | ~40% |
| 98% | ~55% |
| 99% | ~74% |
Moving from 95% to 99% per field looks like four percentage points. At the document level it is the difference between a fifth of your abstracts being clean and three-quarters of them being clean. This is not an argument against document AI — extraction at these rates is still faster and often more consistent than manual keying. It is an argument against reading a per-field percentage as a per-document promise, and for building a review step into the workflow rather than assuming the tool removes the need for one.
Precision, recall, and the field you never see
A single accuracy figure also blends together two failure modes that behave very differently: getting a field wrong, and missing a field entirely. Two metrics keep them apart, and they are worth knowing by name.
- Precision answers: of the values the tool extracted, how many were correct? Low precision means visible errors — a wrong number sitting in a cell where you can see it and question it.
- Recall answers: of the values that were actually in the document, how many did the tool find? Low recall means silent omissions — the option window that was never extracted, so it never reached your calendar.
For a CRE firm, low recall is the dangerous one, because a missing field is invisible. A wrong rent figure looks wrong and gets caught. A renewal option the tool never surfaced looks like a lease that simply has no renewal option, and nobody checks a field that is not there. This is why serious evaluations report an F1 score — the balanced combination of precision and recall — rather than a bare accuracy number. F1 refuses to let a tool hide a recall problem behind high precision. If an accuracy claim cannot separate the two, assume it is not measuring the omissions that hurt most.
Why the blended number flatters the tool
The last thing “95%” hides is that a lease is not a uniform document where every field is equally hard to read. The average is dragged upward by the easy fields.
A lease abstract mixes fields of very different difficulty:
- Header fields — tenant, landlord, premises, execution date. Printed cleanly on page one, extracted correctly almost every time.
- Simple economics — base rent, square footage, commencement and expiration dates. Usually clean, occasionally buried in an amendment.
- Structured clauses — escalations, renewal options, notice windows. Harder, because the value depends on reading a mechanic, not copying a number.
- Bespoke clauses — CAM caps and gross-ups, co-tenancy, exclusives, recapture rights. Written differently in every lease, and where extraction genuinely struggles.
If a tool is 99% accurate on the first group and 80% on the last, and most of the fields it is scored on are header and economic fields, the blended figure lands in the mid-90s while the tool is unreliable on exactly the clauses that trigger disputes and six-figure decisions. The accuracy you care about is not the average — it is the accuracy on your highest-cost fields, which for most firms are the options, the critical dates, and the recovery clauses. Our framework for which fields to extract and why sorts every lease field by what an error actually costs, which is the lens to read any accuracy claim through.
What a realistic expectation looks like
So what should a small firm expect? On clean, digitally native leases, current document-AI platforms and business-tier assistants extract standard fields at accuracy in the mid-to-high 90s. On a realistic stack — scanned 1998 leases with stacked amendments, fax-quality estoppels, riders with hand-marked escalations — accuracy on the hard clauses drops meaningfully, because the system is now fighting degraded reading, context spread across documents, and clauses that reference other clauses.
The honest read is that extraction is strong on standard fields and standard documents, weaker on bespoke clauses and poor scans, and always in need of a targeted human check on the highest-cost fields. That is not a reason to distrust the tool; it is the reason the workflow includes a person. Let the model extract everything, then have someone verify only the fields where accuracy is lowest and error cost is highest — the option windows, notice deadlines, and recovery mechanics. The header and economic fields get a spot check. That is how a firm with no abstraction department produces institutional-grade output: the machine reads at scale, and a person spends fifteen minutes on the five fields that can cost real money.
That discipline — accuracy read honestly, review pointed where the risk lives — is the connective tissue of every reliable extraction workflow, and it sits at the center of our broader playbook for turning a lease stack into structured data. It is also the capability that lets a lean team keep a current, trustworthy portfolio view without a back office — the argument at the heart of our case for how small CRE firms out-operate the giants. Understanding what 95% means is the first step; it tells you exactly where to point the one scarce resource you have, which is human attention.
FAQ
What is extraction accuracy in document AI?
Extraction accuracy is the share of field values a document AI system reads correctly when it converts a document into structured data. Each field the system extracts — tenant, rent, renewal date — is compared to a known-correct value and scored right or wrong, and accuracy is the percentage that match. It is usually reported at the field level, averaged across all fields and all test documents. It is distinct from reading (OCR) accuracy, which only measures whether pixels were turned into the right characters. Extraction accuracy is the number that determines whether your data is trustworthy.
What does 95% accuracy actually mean?
By itself, less than it sounds. It almost always means a field-level average: about 95 of every 100 individual field values matched a correct answer on some test set. It does not mean 95% of documents are error-free. If a tool is 95% accurate per field and a lease abstract has 30 fields, the chance a whole document is fully correct is roughly 0.95 to the 30th power, about 21%. So a strong per-field number can still leave most documents carrying at least one error, which is why review stays part of the workflow.
How is extraction accuracy measured?
Against a human-built answer key. A person reads a set of documents and records the correct value for every field — the ground truth. The AI runs on the same documents, and its output is compared field by field to the key. Accuracy is matches divided by attempts. Two things shape the result: how good the answer key is, and what counts as a match — exact character-for-character, normalized to ignore formatting, or a human judging usability. Because those choices change the percentage, an accuracy claim that names neither is not yet meaningful.
Is OCR accuracy the same as extraction accuracy?
No. OCR (read) accuracy measures whether the system converted the page image into the correct characters — a near-solved problem on clean digital documents. Extraction accuracy measures whether the system correctly identified what each value is: that a date is the renewal notice deadline and not the default cure period. Extraction is the harder task and the one that decides whether your abstract is right. If a vendor quotes a very high number, confirm it describes extraction, not reading, because the reading figure is easier to hit and less relevant to your data.
What is the difference between precision and recall?
Precision is how many of the tool’s extracted values were correct. Recall is how many of the values actually present in the document the tool managed to find. Low precision produces visible errors you can catch in review. Low recall produces silent omissions — a field never extracted, so nobody checks it. For commercial real estate, missing an option window or notice deadline (a recall failure) is more dangerous than a visibly wrong number, because you cannot review a field that is not there. The F1 score combines both so a tool cannot hide missed fields behind high precision.
Why is document-level accuracy lower than field-level accuracy?
Because errors accumulate across fields. Field-level accuracy is the per-field average; document-level accuracy is the chance an entire document comes out with every field correct. Each field is another opportunity to miss, so the whole-document rate is the per-field rate multiplied across all the fields. At 95% per field over 30 fields, only about 21% of documents are fully clean. This is not a flaw in the tool; it is arithmetic, and it is the reason a per-field percentage should never be read as a promise about whole documents.
Does higher accuracy mean I can skip human review?
No. Even at 99% per field — better than most tools deliver — a 30-field abstract is fully correct only about three-quarters of the time, so roughly one in four documents still carries an error. The accuracy number tells you where errors are least likely, not that there are none. The workflow that works is to verify by error cost: a person checks the highest-cost, lowest-accuracy fields such as option windows and recoveries, while near-perfect header fields get a spot check. That targets scarce review attention rather than removing the need for it.
How accurate is AI at reading commercial leases in practice?
On clean, digitally native leases, current document-AI platforms and business-tier assistants extract standard fields at accuracy in the mid-to-high 90s. On a realistic stack — scanned older leases, stacked amendments, riders, poor-quality faxes — accuracy on the hard clauses drops meaningfully. Standard fields on standard documents are reliable; bespoke clauses and degraded scans are where accuracy falls and where a human check belongs. The trustworthy way to judge any claim is to run the tool on a sample of your own worst documents before committing.
What accuracy should I require from a document AI tool?
There is no single threshold, because the right bar depends on what a miss costs. Rather than demanding a headline percentage, ask for accuracy broken out on your highest-cost fields — options, notice windows, CAM recoveries — reported separately from easy header fields, and confirm whether the number is field-level or document-level. A tool that is 90% accurate on hard clauses and honest about it is more useful than one advertising a blended 96% it cannot break down. Match the requirement to error cost, not to a round number on a slide.
Key takeaways
- Extraction accuracy is the share of fields read correctly, measured against a human-built answer key; it is not the same as OCR reading accuracy, which only checks characters.
- “95%” almost always means a field-level average. Across a 30-field lease abstract at 95% per field, only about 21% of documents come out fully clean.
- The number depends on the answer key and the match rule (exact, normalized, or human-judged); a claim that names neither is not yet decodable.
- Low recall is the silent danger — a dropped field is invisible in a way a wrong value is not, which is why F1 and per-field breakdowns beat a bare percentage.
- The blended average is inflated by easy header fields; the accuracy that matters is on your highest-cost clauses, and it tells you where to point human review.
Trying to work out what accuracy your firm actually needs, or where your review time should go once you understand the number? That is exactly the kind of question a short working session sorts out against your real documents and workflow. Book your free AI-readiness assessment →
Dirk Jan van Veen, PhD
