Compare

Which document extraction APIs publish evidence for their confidence scores

Sep 4, 2026

Most document extraction APIs return a confidence number with each field. This page records, per vendor, what the number is, what the vendor's own documentation says it means, and whether the vendor publishes a dataset, a sample size or an error curve for it, each with the day the page was read. Velrim is a row in the table, held to the same three questions.

The table

Confidence scores from nine document extraction APIs: the score, the vendor's statement of what it means, and the evidence published, as of 2026-09-04
Vendor	Score returned	What the vendor says it means	Evidence published, as of 2026-09-04
Velrim	One 0 to 1 number per field, always on.	A probability that the field is correct, with the measured gap between claimed and observed published.	Error curves, regenerated on every refit. In the published comparison: 5,378 field answers across 124 public documents, claimed against observed off by 13 points on average, the per-bucket table below, every raw output public.
LlamaExtract	`parsing_confidence`, `extraction_confidence` and a combined `confidence`, 0 to 1, with `confidence_scores: true`.	"On those tiers a score approximates a real probability of correctness". "At a 0.8 threshold roughly 75% of extraction errors fall below the line." Agentic Max and Turbo: "the calibration above does not apply to them."	None found: no dataset, sample size or curve behind the 75% figure on the extensions, configuring or product pages.
Reducto	`confidence` as `"high"` or `"low"`, plus `extract_confidence` and `parse_confidence` 0 to 1, with citations on.	"Low `parse_confidence` suggests OCR errors. Low `extract_confidence` suggests the model was uncertain about interpretation."	None found on the citations, extract overview or response format pages (2026-09-03).
Azure AI Document Intelligence	Confidence 0 to 1 on words, key-value pairs, selection marks, regions and signatures. "not all document fields return a confidence score."	"an estimated probability between 0 and 1 that the prediction is correct. For example, a confidence value of 0.95 (95%) indicates that the prediction is likely correct 19 out of 20 times."	None found: no dataset, sample size or curve behind the 19-out-of-20 sentence on the accuracy and confidence page.
Google Document AI	`confidence` on each entity, 0 to 1.	"Optional. Confidence of detected Schema entity. Range [0, 1]." No sentence on what the number means beyond the range.	None found on the REST reference or the handle-response page.
Amazon Textract	`Confidence` on each Block, 0 to 100.	"The confidence score that Amazon Textract has in the accuracy of the recognized text and the accuracy of the geometry points around the recognized text."	None found on the API reference or the analyzing-documents page.
LandingAI Agentic Document Extraction	A score per chunk with `low_confidence_spans`, threshold 0.95 recommended.	"It is not a probability of correctness. A score of 0.80 doesn't mean there's an 80% chance the output is correct. It's a relative signal."	None found on the announcement post. The post says the score is not a probability.
Mistral Document AI	No confidence field in the annotations documentation.	No statement found.	None.
Gemini 2.5 Flash and gpt-5.4-mini, called directly	No per-field score. gpt-5.4-mini declined `logprobs` on a PDF file input in the comparison's smoke run.	No statement.	Not applicable.

LlamaExtract

Calibrated on Cost Effective, Agentic, and Agentic Plus. On those tiers a score approximates a real probability of correctness, so you can set a threshold directly rather than only ranking fields against each other. At a 0.8 threshold roughly 75% of extraction errors fall below the line.

LlamaExtract docs, Metadata Extensions, read 2026-09-04

Agentic Max and Turbo return scores from an earlier model. They are still useful for ranking fields, but the calibration above does not apply to them.

LlamaExtract docs, Metadata Extensions, read 2026-09-04

Validate the threshold on your own documents. The right cutoff depends on your document mix and on how costly a missed error is. Start at 0.8, score a sample you have ground truth for, and move it until review volume and escape rate sit where you want them.

LlamaExtract docs, Metadata Extensions, read 2026-09-04

Three tiers get a probability claim with one figure attached, and the sentence after it asks the reader to score a sample of their own. As of 2026-09-04, the extensions, configuring and product pages carry no dataset, sample size or error curve behind the 75% figure.

Reducto

By default in v3, numerical_confidence is enabled, which adds granular_confidence with numeric 0-1 scores. extract_confidence: How confident the LLM is about the extraction. parse_confidence: How confident the OCR/parsing is about the underlying text.

Reducto docs, Citations, read 2026-09-03

Low parse_confidence suggests OCR errors. Low extract_confidence suggests the model was uncertain about interpretation.

Reducto docs, Citations, read 2026-09-03

Two numbers and a label, with a sentence on what a low value suggests and no claim that the number is a probability. As of 2026-09-03, the citations, extract overview and response format pages carry no dataset, sample size or error curve.

Azure AI Document Intelligence

Field confidence indicates an estimated probability between 0 and 1 that the prediction is correct. For example, a confidence value of 0.95 (95%) indicates that the prediction is likely correct 19 out of 20 times. For scenarios where accuracy is critical, confidence can be used to determine whether to automatically accept the prediction or flag it for human review.

Azure Document Intelligence docs, Interpret and improve accuracy and confidence scores, read 2026-09-04

Document Intelligence analysis results return an estimated confidence for predicted words, key-value pairs, selection marks, regions, and signatures. Currently, not all document fields return a confidence score.

Azure Document Intelligence docs, Interpret and improve accuracy and confidence scores, read 2026-09-04

A probability claim with a worked example, 0.95 as 19 out of 20. As of 2026-09-04, the accuracy and confidence page carries no dataset, sample size or error curve behind it, and the same page says not every field returns a score.

Google Document AI

Optional. Confidence of detected Schema entity. Range [0, 1].

Document AI REST reference, Document.Entity, read 2026-09-04

A range and nothing on what the number means. As of 2026-09-04, the REST reference and the handle-response page carry no definition beyond the range and no evidence.

Amazon Textract

The confidence score that Amazon Textract has in the accuracy of the recognized text and the accuracy of the geometry points around the recognized text. Type: Float. Valid Range: Minimum value of 0. Maximum value of 100.

Amazon Textract API reference, Block, read 2026-09-04

A 0 to 100 score on every block, defined as confidence in the recognized text and its geometry. As of 2026-09-04, the API reference and the analyzing-documents page carry no dataset, sample size or error curve.

LandingAI

It is not a probability of correctness. A score of 0.80 doesn't mean there's an 80% chance the output is correct. It's a relative signal: 0.80 is less certain than 0.95, which is less certain than 0.99. March 11, 2026.

LandingAI, Introducing Confidence Scores, read 2026-09-04

We recommend 0.95 as a starting threshold. This is the threshold we use internally when evaluating parsing quality. Depending on your use case and risk tolerance, you may want to adjust it.

LandingAI, Introducing Confidence Scores, read 2026-09-04

The one vendor statement in the table that says the number is not a probability. As of 2026-09-04, the announcement carries no dataset, sample size or error curve, and makes no claim that needs one.

Mistral Document AI

As of 2026-09-04, the annotations documentation has no confidence field. In Velrim's published comparison of six extraction setups (September 2026), the Mistral OCR 4 row carries "none surfaced" in the confidence column and 40.3% on absent fields.

Velrim

One number per field, 0 to 1. In Velrim's published comparison of six extraction setups (September 2026), across every field the score graded, the claimed confidence and the observed share correct differed by 13 points on average with refusals counted as a coin flip, 10 points without them. On the 33 values Velrim itself invented, the score averaged 0.40 against 0.64 overall. The curves are on the error curves page and regenerate on every refit. The definition of the fabrication measure is on the metric page.

What Velrim's confidence score said and how often it was right, ten buckets, 5,378 field answers, from the published comparison
Claimed confidence	Mean claimed	Share correct	Field answers
0.0 to 0.1	0.07	0.14	329
0.1 to 0.2	0.14	0.27	212
0.2 to 0.3	0.24	0.41	216
0.3 to 0.4	0.36	0.39	273
0.4 to 0.5	0.44	0.55	354
0.5 to 0.6	0.55	0.43	271
0.6 to 0.7	0.66	0.69	1,152
0.7 to 0.8	0.76	0.75	793
0.8 to 0.9	0.84	0.75	978
0.9 to 1.0	0.93	0.97	800

Each row is the fields whose score fell in that range, the mean score claimed, and the share that were correct. A score that meant what it said would show the second and third columns equal. The rows between 0.1 and 0.5 run under-confident and the 0.8 to 0.9 row runs over-confident. The italic line under the same chart in the published comparison gives the noise floor: at this sample size a score with no error at all would still measure about 8 points off. The raw outputs, the answer key and the scorer are public, DOI 10.5281/zenodo.22233430, and the eval CLI computes the same curve for any adapter on any golden set.

Reading the table

Two vendors say their score is a probability, LlamaExtract on three tiers and Azure for field confidence. One says it is not. The rest define a range. As of 2026-09-04, none of the seven vendors above publishes a dataset, a sample size or an error curve for the number. Velrim publishes all three, and the gap they show is 13 points.