Axis: — (synthesis / capstone) · Prereq: all prior modules · Next: Module 12 (what structure can't tell you) Hook: A frontier model isn't one technique — it's a dozen, composed. Let's take a real one apart, map every technique to the cost (of the five) it buys, and be scrupulously honest about what a decomposition can and cannot tell us.
The 2-minute version (no math)
We've met the techniques one at a time. A real frontier model uses many at once, layered together. Here's one taken fully apart — every technique it carries, sorted by which cost it's paying down.
The crucial honesty: seeing a dozen techniques together tells you what's present, not which one causes which result. We're reading a parts list, not running a controlled experiment. Nobody has removed each technique one at a time and re-measured (that's an "ablation," and we don't have it). So this is a fingerprint — powerful for saying what a model is and where it came from, silent on "this one made it good."
Under the hood — decomposing DeepSeek-V4-Pro
Its fingerprint carries 13+ techniques. Sorted by the cost each attacks:
| Technique | Module | Cost it buys |
|---|---|---|
| (latent KV) | 4 | #3 memory |
| (sparse attention) + attention sinks | 6 | #2 compute |
| + + + norm_topk_prob | 7 | #2 (active-FLOPs ↓), #1 up |
| 8 | #4 decode latency (lossless) | |
| 9 | #1 / #2 / #5 (bits) | |
| 2 | context reach (enables long #3) | |
| swiglu_limit / clamping | 10 (primitives) | stability, not a cost axis |
| muon_optimizer | — | a training-time choice — not inference structure at all |
Two things the table makes visible:
- Orthogonal vs entangled. Precision (fp8) is orthogonal — it composes with anything. MLA and DSA are entangled — both rewrite the attention/KV path, so you can't cleanly credit a KV win to one without the other. A parts list can't separate entangled contributions; only an ablation can.
- Not everything in a "technique stack" is architecture.
muon_optimizeris how it was trained, not how it runs — it shows up in the fingerprint but buys none of the five inference costs. The stack mixes structural and training-time facts; we label which.
The negative exemplar (why technique-count is not a score)
The senior review insisted on this, and it's the most important guardrail in the module:
NVIDIA-Nemotron-3-Ultra-550Bcarries a rich stack too (Mamba-SSM + MoE + MTP + attention), yet its is ~4× (549 B total / 135 B active) versus ~60× forDeepSeek-V4-Pro(1.57 T / 26 B active). Same "MoE + hybrid + MTP" checklist, ~15× apart on active footprint. And the gap is not just "Nemotron routes to more " (it does — 22 of 512 vs 6 of 384): SR is a parameter ratio, and Nemotron keeps a large always-on non-expert core firing every token (its and attention layers), so much more of it is active regardless of . Expert-ratio and param-SR are different numbers; don't read one off the other.- The lesson: a technique checklist is not a score. What separates these two isn't the list of techniques — it's the settings (, active count, layer mix), which the fingerprint does record. So the honest move isn't "count techniques," it's "read the settings." Naive counting is meaningless; the numbers that matter are right there in the fingerprint, just not in the checklist's length.
The honest trade-off (the Verdict)
- A "technique stack" is a fingerprint (co-occurrence), not a causal signature (pre-committed guardrail). It says what a model is — — enabling comparison, deduplication, and provenance. It does not say "technique X made it good"; that's Tier-1 and needs ablations we don't run.
- Where the stack points: lineage hypotheses, to be verified — not history read off the config. When DSA and MLA appear in GLM/Kimi after DeepSeek, or MTP goes from novelty to near-universal, structural similarity suggests adoption. But co-occurrence alone is a hypothesis, not proof that one lab took another's technique — inferring transfer from shared structure would be exactly the co-occurrence→causation error this module warns against. Confirming real technique transfer needs (Module 12), not config similarity. That handoff — structural signal → forensic confirmation — is where a co-occurrence pattern becomes a provenance finding, and it's the intelligence a benchmark leaderboard can't produce.
- So read a stack as identity + a lineage hypothesis to test, never as a scoreboard or a settled family tree.
Glossary delta
technique stack (vs causal signature) · ablation · orthogonal vs entangled techniques · training-time vs inference-structure · co-occurrence vs causation · propagation / provenance
Prev: Module 10 — Topology · Next: Module 12 — What structure does not tell you Evidence: atlas snapshot 2026-08-13. Exemplars: DeepSeek-V4-Pro (stack), Nemotron-3-Ultra (negative).