Axis: precision (orthogonal to everything else) · Prereq: Module 0 (parameter count / memory) · Next: Module 10 (Topology) Hook: Every technique so far changed the model's structure; precision changes the bits per number. Fewer bits → less memory (cost #1/#5) and faster math (cost #2). It's also the technique most buried in myth — "4-bit means worse" — and the myth dies on one distinction.
The 2-minute version (no math)
A model is millions of numbers, and you get to choose how many bits to store each in — 16, 8, or 4. Fewer bits = a smaller, faster model, but coarser numbers (rounding error).
Here's the distinction the myth misses: two completely different things get called "4-bit." One is training the model in low precision from the start — it learns to be robust to the coarseness. The other is taking a finished high-precision model and squashing it down afterward (post-hoc). The first is usually fine; the second depends heavily on how carefully it's done and on how big the model is. So "4-bit ≠ worse" is true — but only with that caveat attached. A guide that drops the caveat is selling something.
Under the hood
- (
fp8_native): the model is trained and/or served in 8-bit float from the start — weights (and often activations) are fp8, and the model adapted to that precision during training. Its error profile is baked in and benign. The DeepSeek line pioneered this at scale. - Post-hoc quantization: take a finished bf16 model and convert it to a low-bit format after training — NVFP4 / MXFP4 (4-bit float formats with per-block scaling) or int4. Quality now depends entirely on method: calibration data, block size, and which tensors are kept at higher precision (attention and embeddings are often spared).
- Not comparable across scale — and for it's the active count that matters. For dense models the intuition is over-parameterization: a larger model carries more redundancy, so the same rounding error does proportionally less damage — bigger tends to quantize more gracefully. But for MoE the capacity that absorbs error is the active parameters per token, not the total: a 550 B-total / 50 B-active MoE behaves, for quantization robustness, closer to a ~50 B model than a 550 B one (getting this backwards — "it's huge, so 4-bit is safe" — is a real trap). Never compare a bit-width across architectures or scales; the same "4-bit" label means different things.
- What our fingerprint sees — and what the tag doesn't say: the format — the
fp8_nativetag, orNVFP4/MXFP4in the name — not the realized quality ( format vs Tier-1 quality). And the tag is coarser than it looks:fp8_nativesays fp8 is the native format, but not whether the model was trained end-to-end in fp8 or bf16-pretrained then fp8-adapted (different error profiles). Likewise NVFP4/MXFP4 can be layered on either an fp8-native or a bf16 lineage — the format label alone doesn't reveal which. So read it as "what precision it ships in," not "how it got there."
Fingerprint evidence
(auto-populated — _evidence/module-09.evidence.md)
| Model | Precision signal | Reading |
|---|---|---|
DeepSeek-V4-Pro-0813 | fp8_native | trained-in fp8 (training-time choice) |
DeepSeek-V4-Pro-NVFP4 | fp8_native + NVFP4 | fp8-native base then re-packed to 4-bit — both layers present |
Kimi-K2.6-NVFP4 | NVFP4 (post-hoc) | a bf16-lineage model quantized after the fact |
57 current models are fp8_native. Read the CN/Flash "4-bit" line through this lens (see
[[reference_quant_fidelity_framing]] internally): much of it is quality-at-4-bit achieved by design
(quality-density), not "4-bit magically beating FP8."
The honest trade-off (the Verdict)
- Which cost: buys cost #1 (VRAM to hold), cost #2 (faster low-bit math), and thus cost #5 (serving). It trades precision-per-parameter against scale.
- The pre-committed guardrail: "4-bit ≠ worse — when the model was trained or adapted for it." FP8-native (training-time) and post-hoc NVFP4 have different error profiles; we never conflate them, and we never quote a "4-bit is fine" claim without saying which kind and at what scale.
- What we have / haven't measured: we record the format (Tier-2). A paired dense-vs-quantized evaluation of the same model is Tier-1 — we cite it where we have it and flag where we're asserting a format fact, not a measured quality result.
Glossary delta
precision / bit-width · FP8-native (training-time) · post-hoc quantization · NVFP4 / MXFP4 · block scaling · precision-per-parameter vs scale · quality-density
Prev: Module 8 — Decoding acceleration () · Next: Module 10 — Topology Evidence: atlas snapshot 2026-08-13. Exemplars: DeepSeek-V4-Pro (native + NVFP4), Kimi-K2.6-NVFP4.