Benchmark
Software-engineering benchmark (version 1.1) for scoring AI models on coding-agent tasks; labs such as DeepSeek and Z.ai cite its scores in model releases.
Flash's off-peak input price is under a quarter of what V4-Pro cost, and on DeepSeek's own table it beats the old Pro checkpoint on Terminal-Bench, but it scores 36.8 on Humanity's Last Exam and no V4.1-Pro has a date.
Reality
The rate undercuts the cached-input prices OpenAI and Anthropic publish by more than 130 times, and DeepSeek's own release says the sparse-attention design behind it has untested limits at cache boundaries.
Reality
The 5 trillion token ceiling only binds if every allocation is claimed and then spent. Until it is, this is a short window to test a rival coding agent on someone else's inference bill.
Reality
Grok 4.6, Gemini 3.7 Flash, DeepSeek V4 Pro and GLM-5.3 all chase agents that stay on task. The pricing underneath them is moving faster than the benchmarks.
Reality
Z.ai says its 743B-parameter GLM-5.3 hits 34.5% on its own code bench using 22% fewer output tokens than GLM-5.2. The weights are still two weeks out.
Reality
GitHub added xAI's model to Copilot on August 14 across eight developer surfaces at usage-based pricing. The benchmark case, including xAI's own terminal scores, is mixed.
Reality
Z.ai's August 14 post claims post-training gains for coding agents, but the company's release notes still stop at GLM-5.1 and there is no API endpoint, model identifier or weight download.
Reality
Z.ai says all of GLM-5.3's coding gains came from post-training on tenfold more long-horizon task environments. The uneven benchmark jumps tell you where that money actually landed.
Reality