Accuracy claims are worthless without a denominator. This page publishes the metric definition, the document mix, the adjudication process, the per-category results and — more usefully — the categories where DocxIntel degrades and by how much. Read it sceptically, then reproduce it on your own documents.
Measured on the published DocxIntel benchmark set with a fixed extraction schema per document category, scored as exact match on normalised values against human-adjudicated ground truth. Aggregate accuracy is weighted by field count, so it is not the arithmetic mean of the per-category figures below.
Most published accuracy numbers are unfalsifiable because the metric is never defined. Here is ours, in enough detail that you could argue with it.
The unit of measurement is an extracted field value — a policy number, an invoice total, a patient date of birth — scored as a whole. One wrong character fails the whole field. Character-level accuracy would produce a flattering number and tell you nothing about how much rework lands on your team.
A field is correct only if the normalised value is identical to ground truth. Normalisation handles case, whitespace, date formats, separators and Arabic-Indic numerals. There is no fuzzy matching, no edit-distance tolerance and no partial credit for getting most of an address right.
Ground truth was keyed independently by two annotators per document and every disagreement was adjudicated by a third reviewer against the original page. No model output was used to seed, suggest or verify ground truth at any stage.
All 96,300 field instances count, including fields the model left empty and pages a human found hard to read. Nothing was excluded as out of scope, illegible or unsupported. Silent exclusion is how accuracy numbers get inflated, so there are no exclusions.
The benchmark set is held separately from the data used to develop and tune the shipped models, and results are reported from a single scoring run per release rather than the best of several attempts.
This is what DocxIntel achieved on this set. It is not a warranty for your documents. The only accuracy figure that binds us commercially is the threshold agreed in writing against your own document set in a Proof of Value.
The set was built to over-represent the inputs that break extraction tools, not the clean native PDFs that make demos look good. Native digital documents are less than a quarter of it.
Documents were sourced from anonymised and synthetically generated sets representative of UAE and GCC insurance, banking and healthcare paperwork. No customer document is used in the published benchmark.
The credibility of any accuracy number lives entirely in how the reference values were produced. This is the process, including the parts that were expensive.
Each document category was assigned a fixed extraction schema with an explicit definition for every field, including how to handle absent values, multiple candidates on one page and continuation fields. Schemas were frozen before a single document was scored, so no field definition could be adjusted to suit a result.
Every document was keyed independently by two annotators working from the original page image with no access to model output and no access to each other. Bilingual documents were keyed by annotators fluent in both Arabic and English.
Where the two keyed values differed, a senior reviewer adjudicated against the source page and recorded the reason. Disagreement ran at roughly 4% of fields overall, concentrated in handwritten Arabic and fax categories — a useful signal in itself, because it marks the fields humans also find hard.
Where all three humans agreed a value could not be read from the page, the field was marked illegible and kept in the set with a null ground-truth value. The model is then scored on whether it also declines to populate it — inventing a plausible value for an unreadable field is a failure, not a near miss.
Scoring is a deterministic script: normalise both values with the documented rules, compare for exact equality, aggregate by category and by field. The same script runs during a Proof of Value against your adjudicated set, so the two numbers are directly comparable.
Reporting a single blended number would hide the only thing an evaluator needs: which of your document types will land in a review queue. The residual error column names the failure mode that dominates each category.
Native PDFs and office documents
Bilingual Arabic-English tables
Stamped and signed invoices
Phone-camera photographs
Arabic handwritten forms
Multi-generation fax copies
Aggregate, weighted by field count
Scored on the benchmark set described above, as exact match on normalised values against human-adjudicated ground truth, with every schema field in the denominator. The aggregate is weighted by field count rather than by document count, which is why it sits above the mean of the column. Fields routed to review are those below the default confidence threshold; they are counted as errors in the accuracy column if the value was wrong, whether or not they were flagged.
A vendor that cannot tell you where its product degrades has either not measured it or would prefer you found out after signing. Handwritten Arabic at 98.1% and multi-generation faxes at 97.2% are the honest edges of this benchmark, and both behave predictably rather than catastrophically.
This is the useful test of any accuracy claim. Ask a vendor for their worst category and the failure mode that dominates it. If they will not answer, the headline number is not measured the way yours would be.
An accuracy figure only becomes an operating model when confidence is calibrated — that is, when a score of 0.9 really does mean roughly nine out of ten right. Calibration is what lets you size a review team instead of guessing.
Confidence is calibrated against observed correctness on the benchmark set, so the predicted probability tracks the measured hit rate within a few points across each band. An uncalibrated score cannot be used to set a threshold, however precise it looks.
Thresholds are configured per field, not per document, because an IBAN and a free-text remark do not deserve the same scrutiny. Financial identifiers can sit at a strict threshold while descriptive fields pass through at a looser one.
At the default threshold, 2.3% of benchmark fields route to review and the fields passing straight through score 99.7%. Tightening the threshold raises straight-through accuracy and grows the queue; loosening it does the reverse. The trade-off is explicit and yours to set.
A flagged field arrives with the page, the bounding box and the original image region highlighted, so a reviewer confirms or corrects one value rather than reading the whole document. This is why review cost tracks flagged fields, not flagged documents.
Page number, bounding-box coordinates and confidence travel with each field into your systems, so an internal auditor or a regulator can trace a posted figure back to the pixels it came from months later.
See field traceabilityBecause reprocessing is unmetered, you can re-score your own adjudicated set against a new model bundle before promoting it. Accuracy regression testing on your real documents becomes routine rather than a budget decision.
The published number is evidence of method. The number that should decide your purchase is the one measured on your paperwork, in your environment, and written into a contract.
Between 300 and 1,000 documents spanning your real categories, deliberately including the scans, photographs and handwritten forms your current process struggles with. A set of clean native PDFs will produce a flattering number that tells you nothing.
Fields, field definitions, normalisation rules and the treatment of absent and illegible values are fixed before anything runs — using the same rules published on this page. Both sides sign the definition of correct before either side sees a result.
Your named reviewer adjudicates the reference values, because ground truth on your documents is a domain judgement, not a vendor one. We supply the tooling and the adjudication workflow; you decide what the right answer is.
DocxIntel runs on your hardware behind your firewall, the same deterministic scoring script runs against your adjudicated set, and results come back per category and per field — including which fields would have been routed to review at each threshold.
The Proof of Value states the accuracy threshold on your set before the work begins. Meet it and the licence converts at the price already agreed; miss it and you keep the measured report and owe nothing further. That is the only accuracy promise we are willing to make.
If a question you need answered is missing, ask an engineer directly rather than a sales team.
No, and the difference matters enormously. Character-level OCR accuracy of 99% means roughly one wrong character in every hundred, which on a fifteen-character IBAN is a broken IBAN most of the time. Field-level accuracy of 99% means 99 of every 100 extracted field values match the human-adjudicated ground truth exactly after normalisation. It is the harder metric, and it is the one that maps to how many records a human has to fix.
