State of AI Report 2026, the ninth edition, says changing GLM-5.1's setup lifted its coding success from 52.5% to 65.5% without retraining. For teams setting AI budgets, the report puts much of the return in setup and user skill, though its introduction's main evidence for that is one 100-task study.
Publishers:nathanbenaich.substack.com · press.airstreet.com · stateof.ai
Perspective Coverage
3 publishers
- Builder
- Builder 43%
- Operator
- Operator 27%
- Investor
- Investor 30%
Reality
- Evidence35
- Adoption
- Not scored
- Hype gap+25
- Incentives60
- Confidence55
buildOne report1 publisher Claude 4.5 Opus builds sub-agent spawning into the model, according to a dev.to write-up
Anthropic launched Claude 4.5 Opus on October 9 with sub-agent spawning in the model and a 500,000-token context, per a dev.to write-up. The claimed SWE-bench Verified lead over GPT-6.1 Sol and Gemini 2.5 Ultra comes from that post alone.
Reality
- Evidence15
- Adoption
- Not scored
- Hype gap+60
- Incentives
- Not scored
- Confidence25
Reflection AI says its first model, Beam, performs at the level of Z.ai's GLM-5.2 while using three to four times less compute than similar open models. Until outside testers report, those are the company's figures, and they count most with buyers who cannot use Chinese models.
Perspective Coverage
12 publishers
- Builder
- Builder 37%
- Operator
- Operator 28%
- Investor
- Investor 35%
Reality
- Evidence35
- Adoption10
- Hype gap+45
- Incentives75
- Confidence60
buildOne report1 publisher MLPerf Inference v6.1 starts timing a four-model RAG pipeline end to end
MLCommons' MLPerf Inference v6.1 adds tests that time a whole RAG pipeline and a 1,007-turn coding-agent replay. Those scores help a buyer size hardware only where the buyer's own pipeline spends its time the way the four-model reference does.
Reality
- Evidence45
- Adoption35
- Hype gap+25
- Incentives
- Not scored
- Confidence45
buildConfirmed2 publishers Microsoft plans to run MAI Code 1.1 Flash on developers' machines in GitHub Copilot, with a limited rollout starting by end-October 2026. Its figures come from one high-end laptop and local pricing is undisclosed, so teams cannot yet size the hardware or the bill.
Reality
- Evidence55
- Adoption12
- Hype gap+20
- Incentives55
- Confidence58
Microsoft's Agent Lightning trains AI agents inside the harness they ship with
Microsoft Research Asia open-sourced Agent Lightning v1.0, a 3,500-line framework whose recipe lifted a 9B model's SWE-bench Verified score by 14.6 points. It runs reinforcement learning on the agent a team already deploys, sitting as a proxy between that agent and its model.
Reality
- Evidence40
- Adoption
- Not scored
- Hype gap+15
- Incentives60
- Confidence40
buildOne report1 publisher AgSpec nearly doubles speculative draft length by indexing code the way coding agents write it
KAIST and Seoul National University's AgSpec nearly doubles accepted draft length for coding agents by indexing files in the diff and JSON forms agents emit. It lives entirely in the retrieval index and leaves model weights untouched.
Reality
- Evidence45
- Adoption
- Not scored
- Hype gap+15
- Incentives
- Not scored
- Confidence40
Google's open-source RRSI rewrites agent scaffolding to add six points on Terminal-Bench
Google Cloud AI Research has released RRSI, an Apache 2.0 tool whose self-rewriting agent harness lifted Terminal-Bench 2.1 scores from 74.2% to 80.2%. Any team can use it commercially, though on tasks the agent never trained against the reported gain falls to 4.7 points.
Reality
- Evidence45
- Adoption
- Not scored
- Hype gap+15
- Incentives50
- Confidence40
buildConfirmed3 publishers Fireworks' Ember-1 used 23% fewer reasoning tokens than Kimi K3 in The New Stack's tests, yet Kimi on the cheapest host would cost $1.96 to Ember's $2.48. Ember beats Fireworks' own Kimi rate and loses at the cheapest, so buyers have to price the host before the model.
Perspective Coverage
3 publishers
- Builder
- Builder 52%
- Operator
- Operator 30%
- Investor
- Investor 18%
Reality
- Evidence55
- Adoption30
- Hype gap+25
- Incentives70
- Confidence58
buildOne report1 publisher Asking GPT-5.6 Luna to name an amphibian flags benchmark transcripts with black-box access
GPT-5.6 Luna says "frog" 70-95% of the time when asked for an amphibian after capability benchmarks, against 12-38% after real use, a LessWrong post reports. Anyone with black-box access can run the check, though its authors cannot yet say whether it detects evaluation awareness or lexical cues.
Reality
- Evidence45
- Adoption
- Not scored
- Hype gap+10
- Incentives
- Not scored
- Confidence40
buildConfirmed2 publishers Broken answer keys and graders that punish correct tool calls drove the verdicts. Epoch AI says it stops each review once it has enough evidence, so the published defect counts are floors.
Reality
- Evidence65
- Adoption
- Not scored
- Hype gap+5
- Incentives35
- Confidence62
buildOne report1 publisher A rerun of the same SWE-bench task can consume 30 times the tokens
A study of eight frontier models on SWE-bench Verified puts agentic coding at roughly 1,000 times the token cost of code chat, dominated by input, with the models' own pre-run estimates correlating no better than 0.39.
Reality
- Evidence60
- Adoption
- Not scored
- Hype gap+15
- Incentives30
- Confidence55
buildOne report1 publisher Anthropic's $200 agent run bought 18 times the runtime of the $9 attempt
A dev.to post credits Anthropic with running the same model and the same prompt under two harnesses, 20 minutes and $9 for a broken result against six hours and $200 for a working one. The hourly spend barely moved.
Reality
- Evidence25
- Adoption
- Not scored
- Hype gap+30
- Incentives45
- Confidence40
buildOne report1 publisher Fixing the deployment target splits the flash-tier coding leaderboard into three winners
GLM-5.3-Flash leads agentic terminal work, DeepSeek V4 Flash is billed as the cheapest per token, and a 2.52B MiniCPM5-2B runs locally under Apache 2.0. The comparison flags most of those numbers as vendor-reported.
Reality
- Evidence34
- Adoption27
- Hype gap+26
- Incentives58
- Confidence41
buildOne report1 publisher Grading a trace twice separates where an agent broke from whether the task finished
NVIDIA's developer blog sets out the five-level rollup from step to benchmark and the two scores that read the same trace. Step-level says where the chain broke. End-to-end reads the environment and says whether the refund posted.
Reality
- Evidence55
- Adoption20
- Hype gap+12
- Incentives50
- Confidence60
buildOne report1 publisher Mystery model Union Alpha hit a billion tokens a minute before vanishing from listings and being revealed as Pareto
OpenRouter has dropped the stealth listing. The model is Pareto 26.9, from unbiased.ai, at $2.50 per million input tokens before a 10 October launch. That is double what 26.8 charged.
Reality
- Evidence64
- Adoption42
- Hype gap+45
- Incentives68
- Confidence55
buildOne report1 publisher Anthropic prices its newer Sonnet a third below Sonnet 4.5
Sonnet 4.5 still leads GPT-5 on the coding leaderboards, and GPT-5 lists about 46 percent below it on a 5:1 token mix. Anthropic's current Sonnet undercuts both of Sonnet 4.5's list prices, and that complicates a routing plan built on the older pair.
Reality
- Evidence40
- Adoption20
- Hype gap+15
- Incentives55
- Confidence45
Rule-based elision before summarization led on efficiency among context-management strategies
Researchers held a coding harness's execution loop fixed and varied planning, tools and context management across four models, and found that what each component is worth depends on how strong the model already is.
Reality
- Evidence52
- Adoption
- Not scored
- Hype gap+12
- Incentives55
- Confidence58
buildOne report1 publisher Claude Sonnet 4.5 follows a constraint on its own reasoning trace 2.7% of the time
A new evaluation gives models explicit rules about what may appear in their reasoning and scores whether they comply while still solving the problem. Compliance rose with model size and fell with more RL training.
Reality
- Evidence58
- Adoption
- Not scored
- Hype gap−5
- Incentives35
- Confidence60
buildOne report1 publisher Slicing FrontierMath by category leaves about 13 problems behind each error bar
A position paper argues that CLT-based intervals dramatically understate uncertainty below a few hundred datapoints. The specialized benchmarks frontier teams build are already smaller than that before anyone slices them by task.
Reality
- Evidence62
- Adoption
- Not scored
- Hype gap+10
- Incentives30
- Confidence58