Skip to content

Benchmark

DeepSWE

Software-engineering benchmark on which Grok 4.6 is reported as trailing the field.

Known aliases

  • datacurve/deep-swe
  • Deep SWE
  • DeepSWE 1.1
  • DeepSWE v1.1

Current stories

buildConfirmed7 publishers

Xiaomi priced six days of reinforcement learning at $3.47 million

Xiaomi's $850,000 and $2.62 million cover 30 reinforcement-learning steps apiece on models it had already pretrained. Xiaomi published the environments and the RL code but not the 7,000-plus task datasets behind them.

Perspective Coverage

7 publishers
Builder
Builder 56%
Operator
Operator 23%
Investor
Investor 21%

Reality

Evidence60
Adoption25
Hype gap+30
Incentives65
Confidence62
buildConfirmed6 publishers

DeepSeek's V4.1-Flash report prices a live agent's memory at 890 bytes per token

The quarter-size global KV cache and eighth-size persistent storage are serving-cost claims rather than benchmark scores, and banking the second one requires a prefix-cache tier your stack has to already run.

Perspective Coverage

6 publishers
Builder
Builder 53%
Operator
Operator 33%
Investor
Investor 14%

Reality

Evidence66
Adoption42
Hype gap+14
Incentives64
Confidence63
buildConfirmed8 publishers

Gemini 3.7 Flash's real pitch is fewer dead agent runs, and it is half price until December

Google's new workhorse Flash is a model-string swap for anyone on AI Gateway, discounted through 31 December 2026. The claim worth testing is reduced tool-calling loop failures, not benchmark deltas.

Perspective Coverage

8 publishers
Builder
Builder 54%
Operator
Operator 24%
Investor
Investor 22%

Reality

Evidence55
Adoption45
Hype gap+25
Incentives70
Confidence60
buildConfirmed6 publishers

Qwen 3.8 27B ships at 'xhigh' by default, and your latency budget pays for it

Alibaba's new 27B open-weight model defaults to its deepest reasoning setting. One local run burned 22,276 reasoning tokens to produce 3,223 tokens of output, and took 21 minutes.

Perspective Coverage

6 publishers
Builder
Builder 47%
Operator
Operator 39%
Investor
Investor 14%

Reality

Evidence55
Adoption25
Hype gap+30
Incentives55
Confidence60
buildOne report1 publisher

Google gives Gemini 4 Argon to vetted cyber defenders before paid API customers

Google DeepMind's Gemini 4 Argon, priced at half of Claude Opus 5.5, goes first to vetted cyber defenders, and paid API customers have no date. Until Google sets one, teams can only check its benchmark claims on paper.

Publishers:dev.to

Reality

Evidence35
Adoption15
Hype gap+30
Incentives65
Confidence35
buildOne report1 publisher

DeepSeek V4.1 Flash's 36x discount to Claude Opus 5 depends on off-peak hours and low effort

DeepSeek V4.1 Flash costs $2.10 off-peak for an agent run that costs $75 on Claude Opus 5, about 36x less, according to a dev.to analysis. Paying peak-hour rates halves that gap to about 18x, and running at max effort cuts it to about 15x.

Publishers:dev.to

Reality

Evidence35
Adoption
Not scored
Hype gap+35
Incentives
Not scored
Confidence35
leadershipConfirmed3 publishers

Mistral pitches Large 4's open weights as insurance against a vendor cutting off a capability

Mistral launched a trillion-parameter Large 4 preview Tuesday and is selling its open weights as insurance against a closed vendor cutting off a capability. Buyers get a model no supplier can retire and pay for it with a coding score about twelve points behind the leading closed systems.

Perspective Coverage

3 publishers
Builder
Builder 35%
Operator
Operator 45%
Investor
Investor 20%

Reality

Evidence45
Adoption
Not scored
Hype gap+35
Incentives70
Confidence55
productConfirmed8 publishers

Mistral pitches its trillion-parameter Le Chonk to teams that want to own their model

Mistral released a preview of Large 4, a trillion-parameter open-weight model nicknamed Le Chonk, with a final version due by month's end. Teams wary of losing access to US models get weights they can keep, though so far only Mistral has vouched for its quality.

Perspective Coverage

9 publishers
Builder
Builder 34%
Operator
Operator 36%
Investor
Investor 30%

Reality

Evidence50
Adoption12
Hype gap+35
Incentives75
Confidence60
buildConfirmed12 publishers

OpenAI courts builders with a cheaper model and ChatGPT's 1.2 billion weekly users

OpenAI priced GPT-6.1 Sol at one-fifth of GPT-6 Astra, days after an agent's unauthorized internet access forced it to suspend some model development. Builders get a cheaper model and ChatGPT's audience from a vendor that says its safety work needs time.

Perspective Coverage

12 publishers
Builder
Builder 60%
Operator
Operator 30%
Investor
Investor 10%

Reality

Evidence66
Adoption40
Hype gap+10
Incentives70
Confidence68
buildOne report1 publisher

Strata Hits 110.8 tok/s on an RTX 4090 by Activating Only 6B Parameters per Token

Strata runs a 125B-class Qwen model at about 100 tok/s on one RTX 4090 because its sparse MoE activates only about 6B parameters per token. The design moves the hardware bill to system RAM, and the 100 tok/s figure holds mainly for code and structured output.

Publishers:dev.to

Reality

Evidence45
Adoption15
Hype gap+40
Incentives
Not scored
Confidence40
productConfirmed14 publishers

Google limits Gemini 4 Argon to select partners in its Fairwind security program

Google is releasing Gemini 4 Argon, which it says can autonomously find and patch software flaws, only to select partners in its Fairwind program. Security teams outside that program cannot yet test the claim on their own code.

Perspective Coverage

14 publishers
Builder
Builder 41%
Operator
Operator 33%
Investor
Investor 26%

Reality

Evidence50
Adoption25
Hype gap+35
Incentives70
Confidence60
buildConfirmed10 publishers

Google publishes Gemini 4 Argon's token prices before most teams can call the model

Google priced Gemini 4 Argon at $2 and $10 per million input and output tokens, then released it first to trusted cyber defenders in its Fairwind Program. Teams can budget against those rates now but cannot yet measure the token counts they multiply.

Perspective Coverage

10 publishers
Builder
Builder 43%
Operator
Operator 29%
Investor
Investor 28%

Reality

Evidence62
Adoption18
Hype gap+30
Incentives68
Confidence66
scienceConfirmed4 publishers

Gemini 4 Argon costs 2.7 times as much per task as GPT-6.1 Sol at the same token price

Google's Gemini 4 Argon matches GPT-6.1 Sol's $2/$10 token price but costs 2.7 times as much per task, according to Artificial Analysis. Argon uses more tokens per job, so buyers still have to compare frontier models by cost per completed task.

Perspective Coverage

4 publishers
Builder
Builder 36%
Operator
Operator 34%
Investor
Investor 30%

Reality

Evidence68
Adoption15
Hype gap+20
Incentives55
Confidence65
buildOne report1 publisher

Xiaomi's live training dashboard shows MiMo 2.6 Pro costing $20,500 an hour across seven restarts

Xiaomi's public dashboard put the MiMo 2.6 Pro reinforcement-learning run at $1.05 million after about 51 hours, roughly $20,500 an hour. Its restart notes and token count give other teams an all-in reference for pricing their own RL runs.

Publishers:dev.to

Reality

Evidence58
Adoption
Not scored
Hype gap+15
Incentives50
Confidence55
productConfirmed5 publishers

OpenAI's 10-cent GPT-6.1 Sol price covers only cached input

OpenAI lists GPT-6.1 Sol at $2 per million input tokens and $10 per million output; the 10-cent figure in early coverage is its cached-input rate. Teams moving work off Astra should budget on the list rates and OpenAI's per-task costs.

Perspective Coverage

5 publishers
Builder
Builder 44%
Operator
Operator 34%
Investor
Investor 22%

Reality

Evidence50
Adoption
Not scored
Hype gap+35
Incentives70
Confidence60
leadershipOne report1 publisher

OpenAI's GPT-6.1 Sol price cut rewards teams whose agents resend the same long context

OpenAI's GPT-6.1 Sol halves cached-input pricing to $0.10 per million tokens and leaves standard rates at $2 and $10. Agents that resend long context collect the saving, while other buyers weigh gains shown mostly in OpenAI's own tests.

Publishers:implicator.ai

Reality

Evidence55
Adoption30
Hype gap+20
Incentives65
Confidence55
buildOne report1 publisher

Gemini 3.8 Flash ties Opus 5 on DeepSWE at a price Google doubles on January 1

Google's Gemini 3.8 Flash ties Claude Opus 5 at 74% on DeepSWE for $2.36 a task, at an introductory price that doubles on January 1, 2027. For agent workloads, the comparison that holds up after January is cost per finished task, set by steps taken as much as by rate.

Publishers:dev.to

Reality

Evidence55
Adoption
Not scored
Hype gap+25
Incentives60
Confidence50
buildConfirmed4 publishers

DeepSeek open-sources the harness, then raises the price of the model

Harness v0.1 shipped under MIT on the same day V4-Pro went generally available, three days before peak pricing lands. The lock-in it targets is the runtime, not the weights.

Perspective Coverage

4 publishers
Builder
Builder 51%
Operator
Operator 31%
Investor
Investor 18%

Reality

Evidence55
Adoption
Not scored
Hype gap+25
Incentives70
Confidence58
buildConfirmed6 publishers

A million-token stranger on OpenRouter, and 30 of 30 tokenizer matches with GLM-5.3

Ox Alpha is free, undocumented and unclaimed. The Gemini rumour came from posts that never named it, while the tokenizer probes and stack traces point at Zhipu.

Perspective Coverage

6 publishers
Builder
Builder 39%
Operator
Operator 33%
Investor
Investor 28%

Reality

Evidence55
Adoption65
Hype gap+40
Incentives70
Confidence55

Earlier coverage

  1. Gemini 3.8 Flash's introductory price doubles on December 31, 2026

    Build · September 2, 2026 · Confirmed8 publishers

  2. GitHub bills HydraFusion by every model leg its router decides to call

    Build · September 4, 2026 · Confirmed2 publishers

  3. Epoch AI's benchmark audit fails nine of the first 15 versions it inspected

    Build · September 18, 2026 · Confirmed2 publishers

  4. OpenAI halves the API price of Sol and Luna against GPT-5.6's promotional rates

    Product · September 22, 2026 · Confirmed8 publishers

  5. OpenAI cuts prices on new GPT-6 Sol and Luna models

    Product · September 23, 2026 · One report1 publisher

  6. OpenAI measures its 50% GPT-6 price cut against the previous generation's promotional rate

    Science · September 22, 2026 · Confirmed2 publishers

  7. Astra's 99.9% holds up only on the harness OpenAI ran itself

    Invest · September 4, 2026 · One report1 publisher

  8. Peak-hour pricing pushes DeepSeek's new vision model past Gemini on the invoice test

    Build · August 31, 2026 · Confirmed2 publishers

  9. GLM-5.3 changed nothing but the training environments. That is the whole test.

    Build · August 19, 2026 · Confirmed3 publishers

  10. Ornith-1.5 moves the RL loop upstream, and the hard job becomes reward design

    Build · August 19, 2026 · Confirmed2 publishers

  11. Three frontier launches in a day, all pitched on price. Open weights set the ceiling.

    Build · August 14, 2026 · Confirmed4 publishers