Optimizer for neural network training that updates weight matrices using orthogonalized gradient steps, often used alongside AdamW on selected parameter groups.
The optimizer behind an October 2024 NanoGPT speed record is now part of the mainstream PyTorch stack. Its scope covers hidden matrix parameters, so the rest of the model still needs a second optimizer.
Reality
Alibaba's Qwen3.8-Flash-Next preview activates 6B of its 125B parameters per token. Per-token compute drops to under a quarter of the dense 27B's, and about 125GB of weights still has to stay on device.
Reality
Hugo Vergnes reports 0.384 CORE from 65.3 billion tokens on eight rented B200s. The $998 buys 43 hours of node time, not the 139 GPU-hours of failed run that produced the recipe, and not the evenings that wrote the framework.
Reality
The efficiency claims are self-reported and measured against DeepSeek's own prior model, but the weights are MIT-licensed and already pulled 1.78 million times, which is what turns a ratio into a number a buyer can carry into a renewal.
Reality
Qwen3.8-Flash-Next puts 36 Gated DeltaNet layers and 12 sparse-attention layers on Hugging Face, which means the retrieval budget Qwen4 will inherit is something you can measure against your own traces now.
Perspective Coverage
5 publishersReality