Key Takeaways
- Frontier models with uncapped reasoning time beat specialized platforms on precision, but they fail silently on recall by dropping table rows and context.
- PDF files were built as print instructions rather than structured data containers, creating structural blind spots for text-only LLMs.
- Reducto built specialized vision models like R1 because production pipelines cannot catch silent data loss without manual verification of every page.
- Document parsing requires an end-to-end vision stack similar to autonomous driving rather than heuristic text extraction rules.
The Hidden Recall Bug in Foundation Models
Most teams building document extraction pipelines run a benchmark on a hundred sample pages, check the outputs, and assume their system works. If an extraction output contains no obvious hallucinations, they ship it. That approach hides an expensive failure mode.
Reducto co-founder and CEO Adit noticed this exact problem while benchmarking foundation models against specialized processing platforms. When you uncap reasoning time on frontier models, their precision looks extraordinary. The text they return is clean and accurate. The problem is what they leave behind.
“We found this really weird trade-off between precision and recall,” (play from 7:24) Adit explains. “And so a lot of these specialized document processing platforms were worse than frontier models when you set uncapped reasoning time. But the failure mode for the frontier models was on recall. So they were silently dropping rows, missing bits of context, like those sorts of things.” (play from 7:27)
In industries like healthcare billing and financial auditing, a false positive is easy to flag because an analyst spots the fake number. A false negative is invisible. As Adit puts it: “If you're thinking about your extraction layer just missing information altogether, that's the sort of loss pattern that you're just not going to find in production. You don't know what you failed to extract in the first place.” (play from 7:45)
Why Documents Demand Vision Instead of Heuristics
The root cause sits in the file format itself. PDFs were never meant to feed autonomous agents. Adobe engineered the standard in 1993 to send instructions to physical printers. A PDF stores instructions on where to place glyphs and lines on an x-y plane. It does not store tables, reading orders, or semantic hierarchies.
“The original context for PDF as a file format was for printers to be faithful to the original,” (play from 2:55) Adit says. “And so it wasn't really designed around this sort of problem of agents are going to read it.” (play from 3:00)
When text extractors parse a PDF, they scrape character coordinates and try to guess which text blocks belong together. If a financial document contains multi-line headers, nested tables, or footnotes, standard extractors jumble the order. When those disordered tokens reach a frontier model, the model optimizes for a coherent answer. Instead of admitting it cannot parse a mangled table layout, it simply drops the difficult rows.
To solve this, Reducto built custom vision models, including their R1 model, treating the entire page as visual space. Adit compares the technical jump to autonomous vehicles: “We liken it a lot to self-driving car research. And that there's been lane assist in cars for a long time, and it's a very different thing to say that your Waymo will drive fully end to end autonomously. That's the sort of bar that we want to get to with Reducto.” (play from 3:14)
Rule-based parsers act like lane assist. They keep the pipeline steady on clean, single-column documents. But when real-world production data throws complex tables and stamps at the parser, you need a visual architecture that interprets the page as a unified image.
What to Do With This
Audit your document pipeline this week by running a raw row-count evaluation across 50 production tables. Compare the exact number of rows in the source PDF files against the row count in your extracted JSON output. If your extraction drops even two percent of line items, switch your input stage from raw text scrapers to a visual document parser.