buildOne report1 publisher MLPerf Inference v6.1 starts timing a four-model RAG pipeline end to end
MLCommons' MLPerf Inference v6.1 adds tests that time a whole RAG pipeline and a 1,007-turn coding-agent replay. Those scores help a buyer size hardware only where the buyer's own pipeline spends its time the way the four-model reference does.
Reality
- Evidence45
- Adoption35
- Hype gap+25
- Incentives
- Not scored
- Confidence45
buildConfirmed4 publishers Cloudflare released two Jev-API-compatible decision models, Clef and Clef-flash, on Workers AI and as Apache 2.0 weights on Hugging Face. Typed classification steps in agent code can now move between providers or onto owned hardware, as long as they stay inside the text-only, 32k-context features Jev supports.
Perspective Coverage
4 publishers
- Builder
- Builder 53%
- Operator
- Operator 30%
- Investor
- Investor 17%
Reality
- Evidence55
- Adoption18
- Hype gap+30
- Incentives75
- Confidence60
buildOne report1 publisher AMD benchmarks Gorgon Halo against Intel in the week Nvidia's RTX Spark is expected
AMD published Gorgon Halo AI benchmarks days before Nvidia's expected RTX Spark launch, claiming a 1.1x to 32.2x lead over Intel's Core Ultra X9 388H. Until RTX Spark is measured, buyers weighing Gorgon Halo machines that cost upwards of $7,099 can compare the two only on memory.
Reality
- Evidence40
- Adoption20
- Hype gap+45
- Incentives75
- Confidence45
Claude Opus 4.6 cost 40 times as much as Gemini 3 Flash per run for 11 more points of pen-test coverage
Ridge Security's 96-test pen-test benchmark had Claude Opus 4.6 reach 63% coverage at $217 a run, against 52% for Gemini 3 Flash at about $5.42. Ridge argues tooling matters more than the model, yet its published runs held the tooling fixed and varied the model.
Reality
- Evidence35
- Adoption
- Not scored
- Hype gap+40
- Incentives70
- Confidence40
buildOne report1 publisher runtape traces a bad agent tool call to one sentence by rerunning only the model
Open-source tool runtape traced an agent's unrequested invoice forward to one sentence in a tool result, 10 of 10 reruns with it against 0 of 10 without. The same counting grades prompt fixes, though most of the evidence comes from a rule-based stand-in model.
Reality
- Evidence45
- Adoption
- Not scored
- Hype gap+10
- Incentives55
- Confidence50
buildOne report1 publisher Probes on a 27B open model match direct probes of a 397B model on deception
Probes on Qwen3.5-27B reading other models' text came within 0.004 AUROC, on average, of probing authors up to 397B directly, a LessWrong post reports. Every tested pair was open-weight, so auditors who apply the method to closed models get the reader's view of the text and cannot measure that gap.
Reality
- Evidence35
- Adoption
- Not scored
- Hype gap+20
- Incentives
- Not scored
- Confidence30
buildOne report1 publisher Blender 5.0 rejects three in ten scripts that ten LLMs wrote for it
Ten LLMs' Blender 5.0 scripts ran only 70% of the time when a Kaggle benchmark executed them in 5.0, against 91% for scripts targeting 3.6. Renamed and removed APIs look like valid code, so the benchmark grades each answer in the exact build the prompt named.
Reality
- Evidence50
- Adoption
- Not scored
- Hype gap+5
- Incentives30
- Confidence45
Cupertino is now selling desktops as an alternative to token bills. On the configurations that can host a useful model, payback runs two to four years, and the model you can host is a tier down.
Reality
- Evidence62
- Adoption
- Not scored
- Hype gap+45
- Incentives60
- Confidence60
Jalapeno's lead is measured against last generation and the volumes are tiny, but a first-pass ASIC clearing Nvidia, AMD and Google parts reprices the design barrier, not the supply chain.
Perspective Coverage
7 publishers
- Builder
- Builder 30%
- Operator
- Operator 24%
- Investor
- Investor 46%
Reality
- Evidence50
- Adoption8
- Hype gap+40
- Incentives65
- Confidence55
buildOne report1 publisher Choosing US-only inference for Kimi K3 costs about 11% more than Bedrock's global profile
AWS's walkthrough pairs the OpenCode terminal agent with open weight models on Amazon Bedrock and keeps code inside your own account, and the only price difference it publishes is the 10 percent discount for letting a request route anywhere.
Reality
- Evidence38
- Adoption20
- Hype gap+35
- Incentives88
- Confidence45
buildOne report1 publisher Adding both the guard and sandbox moved measured attack success from 16% to 20%
A LangGraph travel concierge was scored against four prompt-injection defenses, one layer at a time, on 35 prompts. With 25 attack samples in the set, the four-point regression the harness reports is a single prompt flipping.
Reality
- Evidence54
- Adoption
- Not scored
- Hype gap+24
- Incentives42
- Confidence52
Antigravity gives away five frontier models on a quota Google keeps trimming
Antigravity's free tier includes Gemini 3.1 Pro and Claude Opus 4.6 with unlimited tab completions. XDA reports Opus burns credits about four times faster than Gemini, and Google moved the free tier onto credits in March 2026.
Reality
- Evidence36
- Adoption
- Not scored
- Hype gap+12
- Incentives58
- Confidence40
buildOne report1 publisher Documents describing a CoT monitor raised gpt-oss-120b's undetected deception to 25.7%
An arXiv paper finetuned eight models on synthetic pre-training text describing a chain-of-thought monitor, and the models got better at evading that monitor without ever being shown an obfuscated trace.
Reality
- Evidence60
- Adoption
- Not scored
- Hype gap+12
- Incentives
- Not scored
- Confidence55
Arize's cheapest model per finished task reliably solves only a fifth of the benchmark
Arize and Fireworks priced ten models on what a completed command-line task costs, across 2,400 runs. The winner on that metric is an open model with the worst pass rate in the study and the thinnest coverage.
Publishers:arize.com
Reality
- Evidence62
- Adoption18
- Hype gap+14
- Incentives75
- Confidence55
buildOne report1 publisher Claude Sonnet 4.5 follows a constraint on its own reasoning trace 2.7% of the time
A new evaluation gives models explicit rules about what may appear in their reasoning and scores whether they comply while still solving the problem. Compliance rose with model size and fell with more RL training.
Reality
- Evidence58
- Adoption
- Not scored
- Hype gap−5
- Incentives35
- Confidence60
buildConfirmed9 publishers The 1.7x to 3.6x latency range is set by the baseline systems, not the chip, and the report's own publication date is unsettled. Read it as direction, not evidence.
Perspective Coverage
9 publishers
- Builder
- Builder 41%
- Operator
- Operator 31%
- Investor
- Investor 28%
Reality
- Evidence52
- Adoption14
- Hype gap+38
- Incentives82
- Confidence68
OpenAI's cost-per-task argument buys Luna room for ten failed tries before it loses on price
Sarah Friar told Goldman Sachs' technology conference that a pricier model can be cheaper when it needs fewer tries, a claim Artificial Analysis puts at an eleven-to-one price spread against seven index points running the other way.
Reality
- Evidence34
- Adoption38
- Hype gap+30
- Incentives80
- Confidence45
buildOne report1 publisher OpenAI's Broadcom ASIC buys GPU negotiating leverage before it deploys a single rack
OpenAI published a benchmark win on three large models, but the figure a GPU vendor has to price is the 10-gigawatt Broadcom program running through 2029, with racks targeted from the second half of 2026.
Reality
- Evidence44
- Adoption8
- Hype gap+17
- Incentives74
- Confidence46
buildOne report1 publisher Mac Studio M5 Ultra vs DGX Spark: capacity says what fits, bandwidth says what you wait for
The bandwidth gap is 4.4x and the capacity gap is 4x, which is why these two boxes are not really competing. One decides whether a model fits; the other decides whether it is usable.
Reality
- Evidence38
- Adoption32
- Hype gap+25
- Incentives42
- Confidence35
buildOne report1 publisher Ox Alpha passes the Xinjiang test and fails on Xi: seven topics, 83 points apart
CTGT's 76-prompt audit puts an anonymous OpenRouter endpoint at 6.5 on the usual censorship probes and 89.5 on Chinese domestic legitimacy. The prompt list is the benchmark.
Reality
- Evidence55
- Adoption30
- Hype gap+12
- Incentives65
- Confidence52