Axis: — (synthesis) · Prereq: Modules 3–9 (the parts) · Next: Module 11 (the full stack) Hook: Modules 1–9 gave you the parts. A real model is a specific arrangement of those parts across depth — which layers are attention vs , where the layers sit, how the head hangs on. Same parts, different topology = different model. This is the layer-layout view, and it's where the axes finally compose.
The 2-minute version (no math)
Knowing the ingredients isn't the same as knowing the dish. A model isn't just "has attention, has , has an SSM" — it's a specific stacking order: maybe the first few layers are plain, the middle is mostly cheap SSM layers with occasional real attention, the experts kick in after layer 3, and a little prediction head hangs off the very end.
Two models built from the exact same parts but stacked differently will behave differently. Topology is the assembly order — and modern configs declare it explicitly, so you can read the recipe straight off the fingerprint.
Under the hood
- Uniform stacks (classic): every layer is identical — attention + , repeated N times. Llama, Qwen-dense. The simplest topology; "N layers" says it all.
- Partitioned / hybrid stacks: the layer type varies by position. The config declares a pattern
in
layers_block_type(a per-layer partition) orlayer_types(a mixer overlay). Two shapes to keep distinct (this is exactly what our config-only fingerprinter had to be made partition-aware about):- Nemotron-3-Ultra:
48× Mamba-SSM / 48× MoE / 12× attention— three block types across depth. - Qwen3.8:
69× linear-attention / 23× full-attention(≈3:1) — the recall-vs-speed bet from Module 5, realized as a layout.
- Nemotron-3-Ultra:
- MoE placement: MoE is usually not every layer — the first few layers are often dense "warm-up" layers, with experts only from some depth onward. Where that boundary sits is a design choice you can read.
- The MTP head hangs off the final hidden state — it's not part of the main stack.
- Global/local interleave (Module 3's hybrid) is another topology axis: which layers are windowed vs full.
Fingerprint evidence
The partition string is the topology — nothing else to compute:
| Model | Declared layout | What it encodes |
|---|---|---|
NVIDIA-Nemotron-3-Ultra-550B | 48 Mamba / 48 MoE / 12 attention | SSM-heavy with a thin attention slice for recall |
Qwen3.8-2.4T-A95B | 69 linear / 23 full | ~3:1 linear-to-full attention bet |
gemma-4-31B-it | gqa_swa_hybrid interleave | local windows + periodic global layers |
Reading these correctly is why config-only ingest had to sum parameters and depth per block type (a Mamba layer, an MoE layer, and an attention layer have wildly different parameter costs) — a uniform-stack assumption would mis-estimate both depth and size.
The honest trade-off (the Verdict)
- Topology is where the axes compose. The attention-spine choice (Modules 3–6) × MoE placement (Module 7) × MTP (Module 8) × precision (Module 9) all land as one specific layer layout. If you want to know "what is this model, structurally," topology is the single most complete answer.
- It's , cleanly — the config declares the layout, so this is high-confidence structure.
- Why a lab chose a given layout (e.g. exactly 3:1 linear:full, or MoE-from-layer-4) is design intent — we read the what, not the why, and we don't infer performance from the ratio.
Glossary delta
topology / layer layout · uniform vs partitioned stack · layers_block_type / layer_types ·
block type · dense warm-up layers · MoE placement · hybrid stack
Prev: Module 9 — Precision · Next: Module 11 — Putting it together: the technique stack Evidence: atlas snapshot 2026-08-13. Exemplars: Nemotron-3-Ultra, Qwen3.8, gemma-4-31B.