On October 15, 2026, NVIDIA announced Nemotron 3.5 Bolt 74B-A9B: 74 billion parameters inside a 24GB footprint, pitched as "the #1 open model in its class on PinchBench."
Row four of the same model card reads: Qwen3.6-35B-A3B, 88.07. Bolt, 87.82. A quarter of a point.
Not a fraud story. Both numbers are NVIDIA's own, and all three rows Bolt loses survived into the published card, which is more honesty than vendor tables usually offer. What deserves scrutiny is "in its class," because that phrase decides what the table measures.
The question is narrow: at a hard 24GB, is it better to compress a 120B model to 2 bits, or to run a native 26B to 35B at 4 bits?
1. Ground Rules
Bolt 74B-A9B was compressed from Nemotron 3.5 Super VL (120B-A12B) through Star Elastic, then quantized with QAD: experts at IQ2_XXS, 2.0625 bits per element, 24GB footprint. It is a hybrid Mamba-Transformer mixture of experts, described as LatentMoE with a C-RADIO v4-H vision encoder; text, image and video in, text out, 256K context. It is a vision-language model, not a text-only LLM. Weights, GGUF builds and the LiLiCorr drafter live at nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB; repository: huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.
Qwen3.6-35B-A3B: 35B total, 3B active, multimodal, 262K context extendable to about 1.01M, Apache 2.0, roughly 23GB at 4 bits.
Gemma 4 26B-A4B: about 26.5B total, roughly 4B active, 256K, Apache 2.0, 16.95GB as a 4-bit GGUF — the smallest footprint, so the most room left for KV cache.
Nemotron 3.5 Lightning 30B-A3B: an August release for high-throughput text, included as a hallucination control because it shares Bolt's lineage.
| Item | Bolt 74B-A9B | Qwen3.6-35B-A3B | Gemma 4 26B-A4B | Lightning 30B-A3B |
|---|---|---|---|---|
| Total / active | 74B / 9B | 35B / 3B | about 26.5B / 4B | 30B / 3B |
| Architecture | Hybrid Mamba-Transformer, LatentMoE | Hybrid MoE, multimodal | MoE, multimodal | MoE, text-first |
| Context | 256K | 262K | 256K | Not listed |
| Input | Text, image, video | Multimodal | Multimodal | Text |
| Precision here | IQ2_XXS mixed | BF16 | BF16 | BF16 |
| At 24GB | 24GB footprint | about 23GB at 4-bit | 16.95GB GGUF | Not listed |
| License | OpenMDW-1.1 | Apache 2.0 | Apache 2.0 | Not listed |
The row people misread is precision: Bolt runs at 2 bits while the other three columns are BF16. NVIDIA's rule was that baselines fit "within 24GB of memory with 4-bit quantization," so this is not a matched-precision bake-off but each camp's best effort at a fixed budget — the comparison a buyer actually faces. Our October pieces sorted machines by VRAM tier and by five VRAM-saving routes; this compares routes at one budget.
2. Three Rows Bolt Loses
The most credible part of a vendor table is what it did not delete. Of seven rows on Bolt's card, NVIDIA wins four and loses three.
PinchBench, by 0.25 points. Qwen3.6 scores 88.07 against Bolt's 87.82. At this scale it is a tie, and a tie is enough to falsify the absolute word "first" on the same page that publishes it.
AA-Omniscience accuracy, by 5.27 points. Gemma 4 reaches 22.17 and Bolt 16.90, with Qwen3.6's 19.47 also ahead. This is closed-book recall, exactly where 2-bit packing should hurt: compression preserves the reasoning path far better than memory granularity, so a model still chains steps correctly while losing the fine details those steps operate on.
Non-hallucination rate, by 7.54 points. Its own sibling Lightning scores 68.81 against Bolt's 61.27. Accuracy asks how often the model is right; non-hallucination asks whether it admits ignorance. Bolt trails on both, so past its knowledge boundary it invents plausible answers instead of declining — and an agent carries that invention forward as fact while running the next three steps on top of it.
Read seven rows together and the shape is consistent. Bolt wins the process tasks: BrowseComp 66.75 against 48.74 and 26.30, Terminal-Bench 2.1 at 50.98 against 44.38 and 37.22, GPQA Diamond 86.43 against 83.40 and 79.61, AA-LCR 64.06 against 61.06. It loses the fact tasks — compression's systematic bias, not random error.
3. What "in Its Class" Means
Both statements hold only if "class" is narrower than "all open models." Two readings survive, and they lead to opposite conclusions.
Class as precision class. Bolt is IQ2 at 2.0625 bits while the Qwen column is BF16, and inside the 2-bit category 87.82 has no rival. The claim is not false; it just drops the qualifier.
Class as 24GB footprint. Qwen3.6 at roughly 23GB in 4-bit sits in the same budget, its 88.07 beats 87.82, and the announcement contradicts the company's own table — "first" as marketing, not measurement.
NVIDIA does not say which it means. But the release shows the company knows how to qualify: the other two placements read "#2 WebArena" and "#2 BrowseComp," both carrying "similar memory footprint." Where a qualifier was easy, it was written; on PinchBench it is missing. Treat this as a specification risk: ask what the class is cut by, because by precision and by memory the winner differs.
4. Fitting 74B into 24GB
Change the architecture, not the weights. Star Elastic is NVIDIA's router-guided architecture search and end-to-end compression framework. A budget-conditioned router searches two dimensions at once: how many routed experts to keep per layer, and how wide to leave the MoE intermediate layers. The result is top-K fixed at 9, down from 22 in the parent. Active parameters fall from 12B to 9B, so each token passes through roughly 60 percent fewer experts while total parameters stay at 74B — which is why a 74B model reaches 84 tokens per second on a laptop. It is not doing the math faster, it is doing less math.
QAD recovers the accuracy. Quantization-aware distillation compresses first, quantizes second, then aligns the quantized model's outputs against the parent so both losses are repaired together. NVIDIA published no ablation, so the only evidence is the result: packed to 2 bits, Bolt still scores 86.43 on GPQA Diamond, above BF16 Qwen3.6 at 83.40.
Mixed precision, applied only where safe. FC1 and FC2 weights, routed and shared, go to IQ2_XXS at 2.0625 bits per element; they are the bulk of the parameters and the only part packed to 2 bits. Mamba linear layers go to NVFP4. The lm_head, token embeddings, attention and LatentMoE weights stay at Q8_0. 74B at a flat 2.0625 bits is about 19.1GB of weights, so the remaining 5GB of budget buys the Q8_0 layers, the vision encoder and KV cache. Two bits for the many dull weights, eight for the few sensitive ones, because crushing embeddings or attention breaks the model.
Speculative decoding pulls throughput back. LiLiCorr is a 1.13B drafter excluding shared embeddings and the LM head, trained jointly with the target. With seven drafted tokens and vLLM at concurrency 64, mean accepted length is 4.68 greedy and 4.25 under sampling. At 4.68, one verification step yields four to five tokens, so a meaningful share of that 84 TPS comes from the drafter rather than from the target running fast.
The thesis, supported by citing ParetoQ and Quest: at a fixed memory budget, compressing a large model beats training a small one from scratch. Testable, but unreproduced outside NVIDIA.
5. Speed: Bolt Loses Here Too
Environment first, or the numbers mean nothing: llama.cpp, Windows 11 ARM64, Surface Laptop Ultra, 128GB unified memory, RTX Spark.
| Metric | Bolt, IQ2 + LiLiCorr | Qwen3.6, UD-Q4_K_M + MTP | Gemma 4, UD-Q4_K_M + MTP | Qwen3.8 27B, DFlash |
|---|---|---|---|---|
| AIPerf SPEED-Bench-Coding TPS | 83.99 | 96.63 | 100.56 | 28.26 |
| AA AgentPerf Local replay, s | 600.80 | 453.97 | 480.68 | 1043.28 |
Bolt is third on throughput. At 83.99 against 96.63 and 100.56 it is about 15 percent slower than Qwen3.6 and 20 percent slower than Gemma 4. It clears its own 80 TPS target but does not win.
End-to-end replay is worse. AA AgentPerf Local runs a full agent trajectory: 600.80 seconds against 453.97 and 480.68, which is 32 and 25 percent slower. This carries more decision weight than TPS, because an agent's felt speed is not decode rate but how long one job takes — replay includes prefill, tool-call gaps and multi-turn stitching.
A high score does not mean it runs. Qwen3.8 27B posts Coding 68.1 and Agentic 45.8 in public AA data, far ahead of Gemma 4, yet on the same laptop manages only 28.26 TPS and 1043.28 seconds — 1.7 times Bolt's duration. A dense 27B pushes every token through every parameter, a structural handicap on unified memory; small active counts are the precondition on the edge, not an optimisation.
One layer must be stated: the four columns use different stacks — IQ2 with LiLiCorr, UD-Q4_K_M with MTP, UD-Q4_K_M with DFlash. This is not a precision test but each camp's best effort inside 24GB. Also correct a misreading: "up to 5x faster than comparable models" belongs to Super VL, the 120B-A12B model, not to Bolt. Bolt's target was 80 TPS and it measured 84; it never meant to win on speed. What the table does support is intelligence per gigabyte — and a gigabyte is a ceiling, not a denominator, since 24GB here includes Q8_0 layers, vision encoder and runtime while Gemma's 16.95GB is a weight file with no KV cache.
6. The One Score NVIDIA Did Not Produce
Artificial Analysis' Intelligence Index gives Bolt 21, fourth of 67 models in its comparison class. Two qualifiers: this is second-hand AA data relayed through orcarouter.ai rather than a figure we pulled directly, and it was measured on the actual 2-bit checkpoint rather than a BF16 reference build. That second point matters, since most compression claims are advertised with BF16 numbers.
The absolute value is modest; the meaning is relative — fourth place inside the 24GB constraint. On the same scale Gemma 4 26B-A4B posts Intelligence 16.7, Coding 39.3, Agentic 2.9, GPQA 79.2, HLE 19.3 and Terminal-Bench 13.6.
A cross-check is instructive. Gemma's GPQA appears as 79.61 at NVIDIA and 79.2 at AA, essentially aligned. Its Terminal-Bench appears as 37.22 at NVIDIA and 13.6 at AA — a 23.6-point gap with neither the version suffix nor the harness disclosed. When the same model can differ by 20 points across harnesses, a single rank should never drive a purchase. AA cannot fill Bolt's speed or cost fields either: no hosted API, no per-token price.
7. Three Workloads, Three Answers
There is no "who is first," only what your machine must do.
Air-gapped, maximum single-box intelligence: Bolt. It delivers 21 inside 24GB while carrying 256K context, image and video input, and BrowseComp 66.75. In compliance-bound work local execution is worth many points on its own — medical records, case files and internal codebases cannot go to any API.
Throughput, concurrency, many users: Qwen3.6-35B-A3B or Gemma 4 26B-A4B. With 3B to 4B active the same card holds more concurrent sessions, and 96.63 and 100.56 TPS beat Bolt's 83.99. Qwen3.6's 262K context suits a single-box coding agent; Gemma's 16.95GB leaves the most headroom for cache.
Low hallucination first: Lightning at 68.81, then Gemma at 22.17. These solve different problems — one asks whether the model says "I do not know," the other how much it knows. Bolt is last on both, and in contract diffing that inventiveness becomes an operating cost.
License matters too. Bolt ships under OpenMDW-1.1: use, modification and redistribution free of charge if the license and notices are retained, but rights end if you sue alleging patent or copyright infringement. Qwen3.6 and Gemma 4 are Apache 2.0. Both permit commercial use and OpenMDW-1.1 is looser than most custom open-model licences, yet its termination clause is not Apache 2.0's patent grant. Not legal advice.
Ecosystem. llama.cpp, Ollama, LM Studio, Unsloth and vLLM all support it; hardware spans RTX Spark laptops from October 16 with up to 128GB of unified memory and 1 petaflop, GeForce RTX PCs, DGX Spark, DGX Station and RTX PRO. Bolt E4B 4GB follows on October 23 for voice assistants and embedded agents.
2-bit compression in 2026 is still vendor-self-attested. NVIDIA offered a testable thesis, a table that prints its own losses, and one independent fourth place — better than a slide deck, but unreproduced.
FAQ
Q1: Is Bolt actually #1 on PinchBench? A1: It depends on how "class" is cut. By 2-bit precision, 87.82 has no higher number on the same table. By 24GB memory footprint, Qwen3.6-35B-A3B's 88.07 wins and Bolt is second. NVIDIA uses the first reading without stating it.
Q2: Doesn't 2-bit quantization break the model? A2: It degrades unevenly. IQ2_XXS at 2.0625 bits per element covers only FC1/FC2 expert weights; Mamba layers use NVFP4; lm_head, embeddings, attention and LatentMoE stay at Q8_0; then distillation aligns the result against the parent. The residue shows in recall — AA-Omniscience accuracy 16.90 against Gemma 4's 22.17 — not in reasoning, where GPQA reaches 86.43.
Q3: Will it run on my 24GB card? A3: NVIDIA's 24GB is a footprint claim, and the demo machine is an RTX Spark laptop with 128GB of unified memory. No per-GPU list and no published test on a 24GB discrete card exists, so fitting depends on context length and KV cache. "24GB footprint" is not "a 24GB card is enough."
Q4: It is slower, so why recommend it at all? A4: Because when the budget binds it converts speed into absolute intelligence: 21 against 16.7 inside the same 24GB, at about 20 percent less TPS and 600.80 against 480.68 seconds of replay. For a single-user, air-gapped box that trade usually pays; for many concurrent users the arithmetic reverses.
Q5: Can I use OpenMDW-1.1 commercially? A5: Yes, free of charge with the license and notices retained, but your rights end if you sue alleging infringement. That is looser than custom licences that cap use by revenue, yet the termination clause differs from Apache 2.0's patent grant, so have counsel confirm. Not legal advice.
Related reading
- Local Coding Models: From 8GB GPU to 8-GPU Node: Picking local coding models by memory tier
- Half the VRAM, Nothing Runs: Five Ways Out: Five VRAM-saving routes and how far each reaches
- Mellum2.1 Deploy SOP: Run a 12B Coding Agent on 24GB: Running a 12B coding agent on a 24GB card