Which Document Extraction API Can You Actually Trust?

Unsiloed AI, Extend, Reducto, LlamaIndex, Mistral OCR, and Anthropic's Claude all turn documents into structured data through an API. On clean digital documents they are hard to tell apart, so we didn't test those. We took three documents from the kind of work where extraction errors cost money, a banking table, a medical record, and a holdings sheet.

Unsiloed led the comparison, but what really sets it apart is that it is the only tool that reliably tells you when it is wrong: Reducto stamped fabricated lab values "high confidence", LlamaParse and Mistral silently dropped 124 and 102 of 171 rows from an oversized sheet, and Extend, the closest competitor on accuracy, returned empty confidence fields on every value, so its errors looked exactly like its right answers.

What Each Tool Offers

The specialized document extraction models of Unsiloed, Extend, Reducto, LlamaIndex, and Mistral OCR all provide a similar platform. Put simply, each provides an API that allows you to input a document and get back structured data in the format that you want. The APIs give you endpoints for parsing (turning the whole document into structured markdown) and extraction (pulling out specific fields that match a schema you define). Most of them also return a confidence score that is meant to tell you how sure the model is about each value.

Claude is the exception here being a general-purpose model that accepts PDFs directly, so we treat "send the document and a schema via tool use" as its extraction API, using Claude Sonnet 5. (Whether a general-purpose LLM is enough for document extraction on its own is a question we've dug into separately.)

What We're Testing

Each test document is hostile in a way document extractors are known to struggle with. The 1949 banking table is 65 rows of worn, smudged print where a 1 reads like a 7, and every cell feeds a printed total, so a single misread silently corrupts the dataset. The lab report is faxed at 120 dpi with noise and skew, right at the edge of readable, where OCR has to guess at digits it can't fully see. And the holdings sheet is a 143-megapixel page that blows past the image size limit most parsers have: they either shrink the page until the digits turn to mush or quietly stop reading partway through. On each one we're looking for two things: how much the tool gets right, and whether anything in its response tells you what it got wrong. All tests were run against each provider's production API in July 2026, with the same schema or parse request sent to every tool, and we saved every raw response; the card popups below show the exact requests and what came back.

Each round below shows the source document, Unsiloed's result as the reference, and the five competitors beside it. Every card marks whether the tool warned you when it was wrong; click any card to open the full result and see exactly how we ran each benchmark.

Backfile Digitization: A 1949 Banking Table

The first document is page 389 of the 1951 Statistical Abstract, from the scan hosted on census.gov: "All Active Banks, Summary by States and Other Areas", 65 rows by 9 columns of dollar figures in worn 1950s print. This is a job every financial data vendor knows: turning decades of old scans into a database. (If your backfile is still image-only PDFs, start with making the scans searchable — everything below assumes you're past that step and extracting structured values.)

The input: deposits split into demand and time, states grouped into regions, all of it adding up to printed totals. Those totals are how we score it: 136 sums to check per extraction. The red box marks the Texas row, the one line every tool read differently.

Each API received the same schema, one row per state; the badge on each card below shows how many of the sums it passed. Every tool misread something in this table. The cards show where each one went wrong and what its confidence score said about it, starting with the Texas row in the red box, printed in worn, slanted type, which no two tools agreed on:

Reading across the row: the printed total is 6,710.3 and demand is 6,048.7, so the time cell must be 661.6. Mistral misread it and the total. Reducto invented a number and vouched for it. Claude got both hard cells wrong. Extend and LlamaIndex read this row correctly, but both misread cells elsewhere in the same table: Extend was off by 90 million dollars on Arizona with its confidence fields empty, and LlamaIndex was off by 60 million on Kentucky at the same confidence 1.0 it gave the cells it got right. Unsiloed's 661.8 is off by 0.2, and it is the only reading that arrived flagged. One cell could be luck; the scoreboard above covers 136 of these sums. The flag is what turns a 98% extraction into something you can actually rely on, because the 2% shows up already marked.

Faxed Medical Records: A Degraded Lab Report

Labs still fax. We took a digital lab report (a blood count and a cholesterol panel, 28 values we know exactly), and made two fax copies of it: a moderate one at 170 dpi and a hard one at 120 dpi with noise, skew, and streaks.

The moderate fax matters as a baseline: all six tools scored a perfect 28/28 on it. Nothing below is about a tool being unable to read a fax.

The hard fax. A human can read it with effort, which makes it exactly the document you want a machine to check first.

On the hard fax, LlamaExtract wins on accuracy: it read all 28 values correctly, and most of the field came within an error or two (each card's badge below shows its count). The more interesting question is what each tool told you about its own mistakes, and there the results look very different:

Five of the six tools landed within two errors of each other, so this round isn't about accuracy. Look at what full confidence meant on this one document, though: LlamaExtract said 1.0 about a perfect read, and Reducto said "high" about values it made up. The same signal covered the best result and the worst one, so the signal tells you nothing. And we only know LlamaExtract got 28/28 because we have the original file to check against. A real pipeline doesn't have that; all it sees is the score, and the score says 1.0 either way. Unsiloed is the only tool whose score pointed at its own mistake.

Oversized Pages: A 143-Megapixel Holdings Sheet

The parse round. We tiled four pages of the Dodge & Cox Income Fund's published portfolio holdings into a single 143-megapixel page: 171 bond rows, seven columns, the kind of oversized dense sheet you get whenever documents are stitched together, and a direct test of how much resolution a parser can really handle. (Tables that sprawl past a single page are their own failure mode — see why multi-page tables still break every extraction pipeline.) We built it from the original digital file, so we know exactly what every one of the 684 cells should say.

The input, all four tiles. At full size, every digit is crisp; any parser that shrinks the page to fit its image limit turns them to mush.

Each API's parse endpoint (for Claude, a transcription prompt) returned markdown, and we counted how many of the 684 cells came back:

The thing to worry about here is not a misread digit; it's the row that quietly disappears. If your pipeline is fed the LlamaParse or Mistral output, nothing in it tells you that 124, or 102, of the 171 holdings are missing.

Which Tool Tells You When It's Wrong

Most of these tools return some kind of confidence signal. The real differences are whether it applies to each extracted value, and whether it holds up when the document gets hard — the missing metric in document extraction, as we've argued before. Here is what each API actually gives you:

Unsiloed returned a score on every field we extracted, and the score follows the document. On clean pages it sits at 0.95 and up; on the hard fax the doubtful values drop to between 0.5 and 0.8; on the banking table, four of the document's five lowest scores land on the Texas row, the row whose deposits don't add up. The scores are a guide, not a guarantee: across both extraction rounds one misread still scored high (a cash cell off by six hundred thousand dollars at 0.93), but every other error we found scored 0.72 or lower.

Mistral OCR returns per-word OCR confidence. Two things hold it back: it scores recognized words, not the fields you extract, and it doesn't move on the values Mistral got wrong. On the banking table its lowest word score anywhere was 0.83; on the hard fax the digit it misread scored 0.98, the same as the values it got right. It tells you how sure the OCR is of the characters, not whether the number is correct.

Reducto returns three signals per citation: a high/low flag, a numeric extract_confidence, and a parse_confidence for whole blocks of the page. They disagree with each other. On the banking table, extract_confidence was empty on every value and parse_confidence was a single number, 0.74, repeated across the entire 65-row table. On the hard fax, the flag said "high" on every made-up value while extract_confidence read 0.0 on most of them, and 0.68 to 0.94 on the rest, made-up values included; the one value it got right scored 0.85, below a made-up 0.94. A tool that scores the same invented number "high", 0.0, and 0.28 at once is not telling you when it is wrong; it is asking you to guess which of its three opinions to believe.

LlamaIndex (compared head-to-head with Unsiloed here) returns per-field confidence and citations when you switch them on, and we did. The score barely moves: 1.0 on 648 of the 650 banking-table fields, including every cell in the sums that fail, and 1.0 on all 28 fields of the 120 dpi fax. The only value it scored below 0.95 in roughly 700 fields is blank on the page. Its citations point at whole blocks rather than values: on the banking table every box covers about two-thirds of the page, and on the fax only 7 of the 28 fields came with a box at all.

Extend documents three confidence fields, one entry per extracted value. On the extraction API we used, all three came back empty on every field and every document, even on its most capable processor. Extend's confidence and review scoring appear to live in its Workflows product, which we didn't test.

Claude returns no per-field confidence through the Messages API.

So the difference isn't that only one tool has confidence scores; several return something with that name. Unsiloed's is the only one you could actually build on: a score for every value that drops when the document gets worse, with a citation that points at the exact spot on the page.

What Decides a Production Ready Pipeline

Unsiloed led the banking table, was the only tool to get the giant sheet fully right, and was one value behind the best read of the hard fax. Just as telling: it is the only tool that never had a disaster round. (This tracks with the public benchmarks: Unsiloed ranks #1 on olmOCR-Bench.)

Extend came within a few sums on the banking table, both Extend and Reducto came within one cell on the giant sheet, and LlamaExtract read the hard fax perfectly before LlamaParse dropped three-quarters of the sheet. But the deciding difference is elsewhere:

Only Unsiloed's confidence signal was actually useful. The others' scores were empty, attached to the wrong thing, in conflict with each other, or stuck at 1.0: there is no cutoff you could set on any of them that separates the right values from the wrong ones. None caught Reducto inventing a lab report and marking it "high", and none caught LlamaParse, Mistral, and Claude silently dropping 74%, 73%, and 21% of a dense sheet.

If you want a document extraction pipeline you can trust, run your own documents through Unsiloed's API.