← All benchmarks

TreasuryQA Bench

Professional Work · 2026-10-19

Veris AI's scanned-archive benchmark, built with Reducto: the agent answers 70 questions from a company drive (a Dropbox twin reached over MCP) holding 1,435 image-only scans of U.S. Treasury volumes, 118,524 pages with no text layer. It must find the volume, locate the table through the contents pages, read it with Reducto's hosted OCR server over MCP, and reply with one exact figure, either a value from a table or a sum, percentage or statistic computed from several. A script, not a model, passes the answer only if it lands inside a range built to exclude every known misreading. Every agent answers each question three times, and pass@1 is the share of correct attempts. Rows are a model in a named coding-agent harness, with no reasoning effort stated.

Top models (higher is better)