Best agent in Databricks' live document-reasoning contest scored 63.3%

Databricks ran the first Grounded Reasoning Cup, a live competition in which academic teams pointed purpose-built agents at complex, enterprise-style document collections ^[1]. Stanford won with 63.3 percent accuracy ^[7], which is another way of saying the best system in the room got roughly one question in three wrong ^[16].

The structure is what makes the number interesting. Eleven academic teams from the U.S. and Canada, in groups of two to four, were paired with OpenAI, Anthropic, or Google DeepMind for model access and mentorship, and each team was required to build exclusively on its partner lab's model family ^[3][4]. They had about two months to optimize against OfficeQA, Databricks' public grounded-reasoning benchmark, which the company says reflects economically valuable enterprise workflows ^[5]. On competition day they ran those same systems in real time against OfficeQA Pro V2, a freshly released benchmark whose whole purpose was to test whether the two months of tuning transferred ^[6]. Databricks framed the event around exactly that question: how far do benchmark improvements generalize to similar real-world tasks ^[2].

The answer is partially, and unevenly. The average team scored about 41 percent, the top three exceeded 50 percent, and Stanford's win was about 22 points above the team average and about 35 points above the average frontier-agent offline baseline ^[8][9]. Work the arithmetic backward and that baseline sits somewhere near 28 percent ^[15], so purpose-built scaffolding roughly doubled what an off-the-shelf agent managed. That is a real engineering result. It is not a deployment result.

The harder number is 18.8 percent: questions that no team solved at all ^[10]. Pool every one of the eleven systems, take the best answer available from any of them, and the ceiling is 81.2 percent ^[17]. That residue is not a ranking artifact or a prompt-tuning gap. It is a class of enterprise document question that current agent design, with frontier models and two months of dedicated effort behind it, does not answer.

What separated the leaders was mostly not the model call. Databricks reports the strong systems combined document preprocessing, targeted retrieval, parallel agents, structured tool use, and answer verification ^[11], and that performance depended less on a single call than on the surrounding machinery: how documents were parsed, how evidence was retrieved, how intermediate calculations were done, and how answers were checked before submission ^[12]. Stanford's method, per Databricks, was to trace wrong answers on the public benchmark back to the exact misstep and convert those failure patterns into reusable skills for its Claude Code agent, covering things like table localization and answer formatting ^[13]. That is debugging discipline, not novelty, and it is the transferable lesson for anyone building the same thing internally.

Databricks' own read is that grounded reasoning over enterprise corpora has improved in the seven months since OfficeQA shipped but is still far from solved ^[19][18]. Operators should treat 63.3 percent as the ceiling for a well-resourced team with two months and lab support, on a corpus built to look like their own filings and reports.

Watch whether OfficeQA Pro V2 scores move once teams can tune on it directly, and by how much: that delta is the size of the generalization gap. Watch the unsolved 18.8 percent for a taxonomy of what breaks. And watch whether the next iteration reports per-question cost and latency alongside accuracy, since parallel agents and verification passes are not free.