outputgate checks the structured output an LLM returns with deterministic rules: schema, types, arithmetic across fields, and provenance, which means every value carries a quote of the source and must equal the number that quote prints. No check calls a model.
The example dataset is 120 balance sheets from SEC 10-Q filings, with each filing's own XBRL figures as ground truth. The three cases below are real answers from qwen2.5:14b running locally. Each case shows what the contract decided and, separately, everything that is actually wrong with the answer: lines left out, lines the page does not print, values that differ from the page, and totals that differ from the filing. Pick a case, switch between contract versions, and click any row to find it on the page.
For every failure in the benchmark, with the numbers, see how LLMs fail at SEC 10-Q balance sheet extraction.