Skip to content

Topic

AI Model Benchmarking

The measurement of language model capability through standardised test suites, and the arguments over what those scores predict about real work.

Current stories

buildOne report1 publisher

Grok 4.7 doubles cost per task at an unchanged per-token price

Grok 4.7 keeps Grok 4.6's $2/$6 token price yet costs $3.74 per task against $1.86, by Artificial Analysis' measurement. Teams that budget from the price sheet will undercount agent spend until they measure tokens per task on their own work.

Publishers:dev.to

Reality

Evidence58
Adoption
Not scored
Hype gap+45
Incentives55
Confidence55
scienceConfirmed4 publishers

Gemini 4 Argon costs 2.7 times as much per task as GPT-6.1 Sol at the same token price

Google's Gemini 4 Argon matches GPT-6.1 Sol's $2/$10 token price but costs 2.7 times as much per task, according to Artificial Analysis. Argon uses more tokens per job, so buyers still have to compare frontier models by cost per completed task.

Perspective Coverage

4 publishers
Builder
Builder 36%
Operator
Operator 34%
Investor
Investor 30%

Reality

Evidence68
Adoption15
Hype gap+20
Incentives55
Confidence65
buildConfirmed17 publishers

Opus 5.5 matched Opus 5's puzzle answers for up to 69 percent less at its lower default effort

Opus 5.5 matched Opus 5 on two reasoning puzzles in The New Stack's tests at 43 to 69 percent lower cost. Both ran at default effort, medium on the new model and high on the old, so the saving a team sees depends on the effort level it pins.

Perspective Coverage

17 publishers
Builder
Builder 43%
Operator
Operator 33%
Investor
Investor 24%

Reality

Evidence62
Adoption48
Hype gap+22
Incentives58
Confidence58
buildConfirmed6 publishers

Spark 1.3's index jump lands on the three tests that carry half the score

Meta's fourth Muse Spark in five months gains four points on Artificial Analysis' Intelligence Index, almost entirely in agentic tests, while two scores fall and the tier Meta benchmarked against rivals is still a limited preview.

Perspective Coverage

6 publishers
Builder
Builder 52%
Operator
Operator 26%
Investor
Investor 22%

Reality

Evidence68
Adoption25
Hype gap+30
Incentives65
Confidence70
buildOne report1 publisher

Vals put Hy4 Preview first among open-weight models on code migration at $3.41 a test

The number came out of one harness on OpenRouter at temperature 0.9 with a 64,000-token output cap, and it holds for a bounded translation task while the same model sits mid-table on open-ended terminal work.

Publishers:runtimewire.com

Reality

Evidence57
Adoption28
Hype gap+19
Incentives62
Confidence60