Compare
OpenAI structured outputs from PDFs: the request, the price per page, and the measured fabrication rate
gpt-5.4-mini takes a PDF as a file part on Chat Completions and returns JSON. This page has the exact request Velrim's published comparison of six extraction setups (September 2026) sent it, free-decode and with Structured Outputs, OpenAI's docs on file inputs, schema rules and prices as quoted on the day, and the model's numbers on 124 public documents: accuracy, how often it invented a value for a field the document did not contain, how often it answered a planted field that could not exist, and the difference constrained decoding made. This setup beat Velrim on accuracy in that comparison, and the table says so.
The request
One Chat Completions call, the PDF as a base64 file part, the schema in the prompt text, temperature 0, model pinned to the dated snapshot. This is the body Velrim's published comparison of six extraction setups (September 2026) sent for every document, with a two-field schema in place of the corpus schemas.
The prompt text is the sentence every model in the comparison got, "Extract the following fields from the document. Return JSON matching this schema. Use null for fields not present in the document.", followed by the JSON Schema. The frozen body also carried logprobs: true until the smoke run, when the API declined it with a 403 on a PDF file input. The parameter was removed and the trim recorded in the run manifest, which is why the confidence column for this row reads "none surfaced". The record is in the analysis plan, §12.
Structured Outputs: one more field
The structured variant sends the identical prompt and file and adds a response_format block with the same schema. The model then cannot emit a token that breaks the schema.
Structured Outputs is a feature that ensures the model will always generate responses that adhere to your supplied JSON Schema, so you don't need to worry about the model omitting a required key, or hallucinating an invalid enum value.
Structured Outputs is the evolution of JSON mode. While both ensure valid JSON is produced, only Structured Outputs ensure schema adherence. Both Structured Outputs and JSON mode are supported in the Responses API, Chat Completions API, Assistants API, Fine-tuning API and Batch API.
To use Structured Outputs, all fields or function parameters must be specified as required. Although all fields must be required (and the model will return a value for each parameter), it is possible to emulate an optional parameter by using a union type with null.
Structured Outputs only supports generating specified keys / values, so we require developers to set additionalProperties: false to opt into Structured Outputs.
When using Structured Outputs with user-generated input, OpenAI models may occasionally refuse to fulfill the request for safety reasons. Since a refusal does not necessarily follow the schema you have supplied in response_format, the API response will include a new field called refusal to indicate that the model refused to fulfill the request.
The comparison's frozen schemas list every field as required, set additionalProperties to false and keep every leaf nullable, which is how the docs say to express a field that may be absent under strict: true. The same schema bytes went to both rows, so the only difference between them is the response_format block. One corpus schema, cut to three fields:
What OpenAI says about PDF inputs, the model and prices
PDF files: On models with vision capabilities, such as gpt-4o and later models, the API extracts both text and page images and sends both to the model.
Token usage: PDF parsing includes both extracted text and page images in context, which can increase token usage. In the Responses API, set detail to auto (the default), low, or high to control the amount of visual detail for PDF page images. Before deploying at scale, review pricing and token implications.
File size limits: A single request can include more than one file, but each file must be under 50 MB. The combined limit across all files in the request is 50 MB.
The detail setting only affects PDF page image processing. Text extracted from the PDF is still included. Chat Completions file inputs don't support detail.
GPT-5.4 Mini brings the strengths of GPT-5.4 to a faster, more efficient model designed for high-volume workloads. Input: Text, image. Output: Text. 400,000 context window. 128,000 max output tokens. Aug 31, 2025 knowledge cutoff. Snapshots: gpt-5.4-mini, gpt-5.4-mini-2026-03-17.
Model | Short context input | Short context cached input | Short context cache writes | Short context output. gpt-5.4-mini | $0.75 | $0.075 | - | $4.50
Structured Outputs is available in our latest large language models, starting with GPT-4o. For new projects, start with gpt-6-astra. Older models like gpt-4-turbo and earlier may use JSON mode instead.
The docs describe a PDF page as extracted text plus a page image, both sent to the model, and say that raises token usage. As of 2026-09-03 the pricing page has no line on how a PDF page is billed, so the per-page cost below is the measured figure.
Cost per 1,000 pages
| Per 1,000 pages | gpt-5.4-mini direct | Velrim |
|---|---|---|
| List price for a PDF page | No line on the pricing page | |
| Measured in the comparison, prompt cached | ~$1.0 (measured, prompt-cached) | |
| List price, grounding and confidence included | $20 |
Velrim's published comparison of six extraction setups (September 2026) measured ~$1.0 (measured, prompt-cached) per 1,000 pages over 319 pages, with the repeated prompt prefix billed at the cached input rate of $0.075 per million tokens and the answer at $4.50 per million. The Gemini rows in the same table came to ~$2.2 (measured, token-priced).
What the comparison measured
| Arm | macro-F1 (norm) | fabrication (absent fields) | wrong among the top 90% by confidence | $/1k pages |
|---|---|---|---|---|
| Velrim | 0.691 | 11.5% [6.1, 19.8] | 36.1% [29.9, 44.1] | $20 (list; measured matches) |
| Gemini free-decode | 0.709 | 17.0% [10.1, 26.7] | not requested | ~$2.2 (measured, token-priced) |
| Gemini constrained | 0.697 | 12.8% [7.6, 20.5] | not requested | ~$2.2 (measured, token-priced) |
| gpt-5.4-mini free | 0.735 | 10.8% [5.7, 18.4] | none surfaced | ~$1.0 (measured, prompt-cached) |
| gpt-5.4-mini structured | 0.718 | 10.4% [5.3, 17.6] | none surfaced | ~$1.0 (measured, prompt-cached) |
| Mistral OCR 4 | 0.712 | 40.3% [31.0, 52.5] | none surfaced | $5 (list; measured $5.3) |
124 public documents, 2102 fields, 96 of them absent by the audited answer key. Free-decode gpt-5.4-mini invented a value for 10.8% of the absent fields, interval [5.7, 18.4]. With Structured Outputs, 10.4%, interval [5.3, 17.6]. Accuracy moved from 0.735 to 0.718 macro-F1, the highest two rows in the table, against Velrim's 0.691. Both rows completed 372/372 attempts.
| Arm | fabrication (pooled, n=96) | all-attempted rule | answered when field present | completed/attempted |
|---|---|---|---|---|
| Velrim | 11.5% [6.1, 19.8] | 11.5% [6.1, 19.8] | 88.0% [84.0, 91.2] | 372/372 |
| Gemini free-decode | 17.0% [10.1, 26.7] | 15.6% [9.3, 24.4] | 96.5% [95.0, 97.7] | 328/372 |
| Gemini constrained | 12.8% [7.6, 20.5] | 11.1% [6.6, 18.1] | 96.7% [95.4, 97.9] | 315/372 |
| gpt-5.4-mini free | 10.8% [5.7, 18.4] | 10.8% | 96.1% [94.6, 97.3] | 372/372 |
| gpt-5.4-mini structured | 10.4% [5.3, 17.6] | 10.4% | 95.6% [93.0, 96.9] | 372/372 |
| Mistral OCR 4 | 40.3% [31.0, 52.5] | 40.3% | 96.6% [95.2, 97.9] | 372/372 |
| Arm | probe fabrication | n |
|---|---|---|
| Velrim | 20.7% [15.3, 27.2] | 198 |
| Gemini free-decode | 18.0% [13.1, 23.5] | 183 |
| Gemini constrained | 20.0% [14.6, 26.4] | 165 |
| gpt-5.4-mini free | 37.9% [30.1, 46.3] | 198 |
| gpt-5.4-mini structured | 39.9% [32.5, 48.4] | 198 |
| Mistral OCR 4 | 53.5% [45.5, 61.5] | 198 |
The trap rows are the second fabrication measure: fields planted in the schema that cannot exist in that document type, so any answer is an invention. gpt-5.4-mini answered 37.9% [30.1, 46.3] of them free-decode and 39.9% [32.5, 48.4] with Structured Outputs, against 20.7% [15.3, 27.2] for Velrim. On the other side, the model answered 96.1% [94.6, 97.3] of the fields that were on the page and Velrim 88.0% [84.0, 91.2], so Velrim's lower rates are paid for in blanks.
The same call through Velrim
| gpt-5.4-mini direct | Velrim | |
|---|---|---|
| What comes back | JSON in the shape of the schema. With response_format and strict: true, the shape is enforced as the model writes. | The same JSON, plus a state, a 0 to 1 score and a page location for every field. |
| Fabrication on absent fields | gpt-5.4-mini free: 10.8%. gpt-5.4-mini structured: 10.4%. | 11.5% |
| Planted trap fields answered | 37.9% [30.1, 46.3] and 39.9% [32.5, 48.4] | 20.7% [15.3, 27.2] |
| Accuracy, macro-F1 | 0.735 and 0.718 | 0.691 |
| Answered when the field is present | 96.1% [94.6, 97.3] and 95.6% [93.0, 96.9] | 88.0% [84.0, 91.2] |
| Same document, three runs, answers differ | 19% and 16% of repeats | 36% of repeats, nearly all on nested line-item tables |
| Per-field confidence | None. logprobs was declined on a PDF file input. | One 0 to 1 score per field, error curve published. |
| Price per 1,000 pages | ~$1.0 (measured, prompt-cached) | $20 |
The score is one number per field, 0 to 1, with its error curve on the error curves page and the definition of the fabrication measure on the metric page. Grounding is a page number, a box and the matched text from the PDF's own word map, no second model call. Price is $0.02 per extracted page, prepaid from $5, about 500 pages free with no card.
Which to pick
gpt-5.4-mini with Structured Outputs, when accuracy per dollar is the whole requirement: it took the top two accuracy rows in the comparison at a twentieth of the price. Velrim, when a value needs a number to sort on and a place on the page to check it, and a field that is not on the page must come back empty, which the trap rows measure.