A gated linear-attention layer that compresses sequence history into a fixed-size recurrent state, so memory does not grow with sequence length.
KAUST's NoteBank lifts Gated DeltaNet's mean accuracy on six 1M-token RULER tasks from 1.7% to 70.3% with no added training. The paper's fix for state forgetting is a small archive of verbatim snippets replayed before each question.
Reality
Moonshot AI's Kimi Linear reports 75% less KV cache and 6.3x faster decoding than MLA at 1M tokens, using three linear layers per attention layer. The speedup falls to parity at 4k tokens, so the saving goes to traffic that runs at hundreds of thousands of tokens.
Reality
Alibaba's Qwen3.8-Flash-Next preview activates 6B of its 125B parameters per token. Per-token compute drops to under a quarter of the dense 27B's, and about 125GB of weights still has to stay on device.
Reality
Alibaba's 27B scores 52 on the Artificial Analysis index from a 17GB quantized file. Filling its 262,144-token window needs roughly 16 GiB of KV cache on top of that, so the file size is the smaller half of the sizing question.
Reality
Alibaba's largest open-weight release fits on a single eight-GPU node only because a community four-bit build takes it down to 1.2 TB. AWS publishes the vLLM config for it and skips the price.
Reality
Qwen3.8-Flash-Next puts 36 Gated DeltaNet layers and 12 sparse-attention layers on Hugging Face, which means the retrieval budget Qwen4 will inherit is something you can measure against your own traces now.
Perspective Coverage
5 publishersReality