NVIDIA's Bolt packs 74B parameters into a claimed 24GB by storing them at 2 bits
NVIDIA says its Nemotron 3.5 Bolt 74B-A9B agent model fits in 24GB and will ship with open weights, data and recipes. The footprint, the license terms and the quality claims stay unverified until the weights are live and outside testers have run them.
The Engineer · Build desk
Key figures: 74B parameters stored at 16 bits take about 148GB. Stored at a flat 2 bits they take about 18.5GB. NVIDIA's claimed 24GB is about 5.5GB above that flat 2-bit layout.
| Figure | Value | Note | Claim |
|---|---|---|---|
| 74B parameters stored at 16 bits, about | 148 GB | 9 | |
| At a flat 2 bits, about | 18.5 GB | 10 | |
| Gap to claimed 24GB, about | 5.5 GB | 12 |
What happened
- The release was scheduled for October 15, and the dev.to post covering it says the weights were not yet confirmed live when it was written.
- NVIDIA named llama.cpp, vLLM, Ollama, Unsloth and LM Studio as launch runtimes, plus the Hermes and OpenClaw agent frameworks.
- NVIDIA calls Bolt faster than dense models and smarter than same-footprint MoE models, and the post says no independent benchmarks exist.
- The closed models launched this month are priced at $4/$20 (Muse Wonder) and $2/$10 (GPT-6 Sol, Gemini 4 Argon intro) per million tokens.
Why it matters
- What it costs cost Bolt trades per-token fees for hardware and electricity, but NVIDIA has announced no retail price for RTX Spark, so a break-even against the closed APIs cannot be calculated yet.
- Who has to decide decision A team cannot commit a product to Bolt until the license text is published, and it should pull only from the official listing name because spoofed listings appear fast.
- What's now possible capability If the release matches the announcement, teams can fine-tune on their own data, run fully offline, and inspect the training data and recipes.
- What it limits constraint NVIDIA's PAIR router sends whole jobs to one machine and does not pool memory, so every node that takes a Bolt job needs room for the whole model.
Next story · Product · 8h ago
RTX Spark laptops from six makers run AI agents on the device, starting at $2,599Confirmed2 publishers
Mixture-of-experts saves compute, not storage [5]. Bolt works on about 9B of its 74B parameters for any one token [5]. That is roughly 12% of the total [8]. Compute per token looks like a 9B model. Memory looks like a 74B model, because the inactive parameters still have to be stored [5].
NVIDIA's route to a small footprint is 2-bit quantization with quantization-aware training. The model is trained to expect the compression instead of being squeezed after the fact [6]. That is the right way to attempt it, in our view.
The sizes do not quite close. At the usual 16 bits, 74B parameters take 148GB [9]. At a flat 2 bits they take 18.5GB [10]. NVIDIA claims 24GB [3], which works out to about 2.6 bits per parameter [11]. That leaves 5.5GB beyond a pure 2-bit layout [12]. The dev.to post covering the announcement does not say what fills it, so the model card's per-tensor breakdown is the document to read.
The 24GB figure is also a statement about the weights alone. According to the post, context length, the KV cache, runtime overhead and agent software all add memory, and it advises treating 24GB as the floor [13]. A machine with exactly 24GB free has nothing left for any of them.
On quality, the transfer condition is plain. For the "smarter than same-footprint MoE models" half of NVIDIA's line to hold on your workload, the 2-bit weights would have to hold up on your own tasks. The only test of that so far is NVIDIA's [1]. Bolt has been benchmarked by its maker and not yet confirmed downloadable [4].
In our view, publishing the training data and recipes next to the weights is what makes the 24GB claim checkable [3]. It is checkable only if the release matches the announcement.
What to watch
- Whether the weights appear under an official Hugging Face or Ollama listing, and what license text ships with them.
- Independent benchmarks of the 2-bit model against the closed models and against same-footprint MoE models.
- The model card's per-tensor breakdown for the gap between 18.5GB and 24GB, and measured memory at real context lengths.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence42
- Adoption
- Not scored
- Hype gap+25
- Incentives60
- Confidence58
Perspective Coverage
12 publishers- Builder
- Builder 40%
- Operator
- Operator 35%
- Investor
- Investor 25%
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
NVIDIA's line that Bolt is 'faster than dense models, smarter than same-footprint MoE models' is NVIDIA's own claim; the post says there are no independent benchmarks for Bolt 74B, and that closed models still lead on independent benchmarks.
- [2]
NVIDIA named five runtimes at launch, llama.cpp, vLLM, Ollama, Unsloth and LM Studio, plus the Hermes and OpenClaw agent frameworks.
- [3]
NVIDIA announced Nemotron 3.5 Bolt 74B-A9B on October 13 as a 74-billion-parameter agent model it says fits in 24GB of memory, with the weights, the training data and the training recipes all to be published.
- [4]
The release was scheduled for October 15, one day before the RTX Spark laptops go on sale; as of the post's writing the weights were not yet confirmed live, and all coverage so far was pre-launch.
- [5]
Bolt is a sparse mixture-of-experts model: only about 9B of its 74B parameters work on any one token, so compute per token looks like a 9B model; inactive parameters still have to be stored, so mixture-of-experts saves compute, not storage.
- [6]
The storage approach is 2-bit quantization with quantization-aware training: parameters are stored at 2 bits instead of the usual 16, and the model is trained to expect the compression rather than squeezed after the fact.
- [7]
Open weights let users fine-tune the model for their own agents, run it entirely offline, and inspect the training data and recipes.
- [8]
Active parameters are roughly 12% of the total.
- [9]
74B parameters stored at 16 bits take about 148GB.
- [10]
74B parameters stored at a flat 2 bits take about 18.5GB.
- [11]
NVIDIA's 24GB claim for 74B parameters works out to about 2.6 bits per parameter.
- [12]
The claimed 24GB is about 5.5GB above a flat 2-bit layout of 74B parameters.
- [13]
24GB is NVIDIA's claim about the weights, not a full system requirement; context length, the KV cache, runtime overhead and agent software all add memory, and the post says to treat 24GB as the floor.
- [14]
The exact license terms are still to be confirmed because the weights were not public at announcement, and wrong or spoofed model listings appear fast around popular releases, so the official listing name should be checked.
- [15]
RTX Spark laptops, with up to 128GB unified memory, go on sale October 16, and NVIDIA has announced no retail prices; Bolt itself is free to download, so the user pays for hardware and electricity.
- [16]
The closed models launched this month are priced per million tokens: Muse Wonder at $4/$20, GPT-6 Sol at $2/$10, and Gemini 4 Argon at $2/$10 introductory pricing.
- [17]
PAIR routes whole inference jobs to one machine and does not pool GPU memory, so three PCs with 24GB each do not become one 72GB virtual GPU.
Sources
12 independent publishers whose own reporting we read for this story.
- blockchain.newsNVIDIA adds Nemotron 3.5 Super VL and two edge models - Blockchain.News
2 articles · October 15, 2026
- dev.toNVIDIA Just Open-Sourced a 74B Agent Model That Fits on a Laptop
1 article · October 14, 2026
- gadgetreview.comNVIDIA’s New 120B-Parameter Model Completes Agent Tasks 5x Faster Than Rivals - Gadget Review
1 article · October 15, 2026
- interestingengineering.comNVIDIA’s latest 120-billion-parameter model completes agent tasks 5x faster than rivals
1 article · October 15, 2026
- nvidianews.nvidia.comOpen From Data Center to Desktop: NVIDIA Expands Nemotron Family With Highest Efficiency Super VL and Bolt Models
1 article · October 15, 2026
- runtimewire.comNVIDIA releases a 120B open multimodal model for AI agents
1 article · October 15, 2026
- siliconangle.comNvidia expands Nemotron LLM family with hyper-efficient models for agentic systems
1 article · October 15, 2026
- the-decoder.comNvidia's Nemotron 3.5 Super VL is an open model built to run enterprise AI agents nonstop
1 article · October 15, 2026
- thenewstack.ioNvidia squeezed a 74B agent model into 24GB so it can run on your laptop
1 article · October 15, 2026
- verdict.co.ukNvidia adds Super VL and Bolt to Nemotron 3.5 open model range
1 article · October 16, 2026
- wccftech.comNVIDIA Announces The 120B Nemotron 3.5 Super VL Model For AI Agents And 2 Tiny 4 GB And 24 GB Edge Models Under The Nemotron 3.5 Bolt Umbrella
1 article · October 15, 2026
- zdnet.comNvidia’s new Nemotron 3.5 Super VL is another step forward for US open-weight AI
1 article · October 15, 2026
Topics and entities
Topics
Entities
- Nemotron 3.5 Super VL
- Nvidia
- Ollama
- Hugging Face
- llama.cpp
- Nemotron 3.5 Bolt