Two Things We Got Wrong About Abliterated Weights

In a previous post we described a mechanical limit we’d run into: when you take an abliterated model — one whose refusal behavior has been surgically removed — and try to extract that edit as a portable LoRA adapter, it works on some models and fails on others. We gave a reason. The reason was incomplete, and the only way we found out was by designing an experiment that could prove us wrong.

It did. Twice. Both corrections are below, along with what actually survives — which is more useful than what we originally claimed.

How the claim drifted

The first version came from a single model: a mixture-of-experts base where our extraction produced nothing usable. We concluded that MoE abliteration can’t be captured as a LoRA. That’s an architecture-level claim from n=1, and a reviewer said so.

A third data point sharpened it. A different lab’s MoE model — NVIDIA’s Nemotron Lightning, a Mamba/MoE hybrid — extracted fine. The difference wasn’t the architecture, it was the file layout: Qwen packs its 256 experts into a single 3D tensor ((256, 2048, 512)), while Nemotron stores each expert as its own 2D matrix. A LoRA factor is a 2D matrix, and standard adapter tooling only targets 2D modules — so against a fused 3D block it simply doesn’t attach. Worth being precise here, because we weren’t: that’s a tooling and layout constraint, not a mathematical one. Nothing in the linear algebra forbids it. So we restated the claim as: the blocker is fused vs. unfused expert storage, not MoE.

Better — but still not proven. Those three models differ in lab, size, architecture, and the abliteration run that produced them. Any of those could be doing the work. We had a plausible story and a confounded comparison.

The controlled test — and the prediction we wrote down first

There’s exactly one way to isolate storage: hold the model, the edit, and the extraction method constant, and change only how the weights are viewed. So we took the same Qwen checkpoint, sliced its fused 3D expert tensor into 256 separate 2D matrices, and re-ran the identical singular-value decomposition.

Before running it, we recorded a prediction in a dated commit: rank-32 energy would come out low (below 0.85), like Nemotron’s ~0.75 — meaning expert edits are inherently high-rank and storage was never the real story. Writing it down first is the point; a prediction you form after seeing the number isn’t a test. (An internal timestamp is admittedly something you have to take our word for. It still constrains us — a prediction on the record is one we can be held to.)

0.99, not 0.85. Prediction refuted. Once unfused, the edit is near-perfectly recoverable at rank 32 — the storage format really was the entire blocker for this model, and now that rests on a controlled ablation instead of a cross-model guess.

Removing that blocker also let us look at the edits themselves for the first time. Both of the things we saw contradicted something we’d already written down.

Correction 1: the two edits didn’t touch the same places

We had written that a recovered adapter is “faithful on attention, lossy on experts.” That turns out to describe one model. Checking Qwen directly: self_attn.o_proj has a relative delta of 0.00000 at every layer we sampled — bit-identical to the base at our reporting precision, consistent with an edit that never touched attention at all. The gate/up projections are untouched too. The entire abliteration lives in the expert down_proj.

Nemotron is the opposite shape: a substantial attention edit plus small expert edits. We had taken one model’s profile and stated it as a general rule. In fact which modules an abliteration targets varies by model and by run — which means a verifier can’t assume where the edit will be.

Correction 2: small edits don’t fake a low score

Our second assumption was that Nemotron’s ~0.75 expert energy was numerical noise. Its per-expert deltas are roughly 20× smaller than Qwen’s (0.0012 vs 0.02), and it’s easy to believe a tiny signal just reads as low-rank-incomplete.

That’s a testable belief, so we tested it: take a real Qwen expert edit, scale it down 20× to exactly Nemotron’s magnitude, and re-measure. Energy came back unchanged at 0.9946.

In hindsight that result is forced by the definition. Rank-32 energy is the fraction of total squared singular value captured by the top 32 components — a ratio. Scale the whole edit by any constant and every singular value scales with it, leaving the ratio fixed. So scale-invariance follows from the metric’s construction, and the control confirms our implementation actually behaves that way. We should have reasoned it out before assuming noise; either route kills the hypothesis.

Which leaves the uncomfortable implication: Nemotron’s expert edits are higher-rank than Qwen’s under the procedures that produced them. Magnitude is ruled out. Method is not — the two models were abliterated in separate runs with different removal recipes, and we can’t yet separate the model from the recipe.

What survives: two properties we had been conflating

Strip out what we got wrong and a cleaner distinction is left — one that’s more useful than the claim we started with:

1. Attachability is a storage property. Fused 3D experts → standard adapter tooling can’t attach. Unfused → it can. This is now established by same-model ablation, and it’s a property of the file format — fixable by slicing, as we did.
2. Recoverability is a property of how the edit was made, not of the format. Qwen’s expert edit is rank-32-clean at 0.99; Nemotron’s is not, at 0.75. This one rests on n=2 across models that differ in method as well as architecture, so we can’t yet say whether the driver is the target modules, the removal recipe, or the training dynamics.

Those are different questions with different answers, and our original claim had them fused together about as tightly as the tensor did.

What it changes for verification

If you’re checking whether a model’s safety behavior has been modified, three practical consequences follow:

• Diff the experts, not just attention. On Qwen, a weight diff restricted to attention would have reported zero change on a model whose refusal behavior had been removed.
• Un-fuse 3D expert tensors before you compare. Tooling that silently skips non-2D tensors isn’t finding nothing — it isn’t looking.
• Report per-module rank-r energy, never a single “the edit is low-rank” verdict. One number would have hidden both corrections.

Why publish the errors

Because the pre-registration is what produced them. We predicted below 0.85 and got 0.99; we predicted noise and got scale-invariance. Had we written the prediction after seeing the results, both would have read as confirmations — the story would have been tidier and wrong.

This is also the honest limit of measurement work: a metric that can’t contradict the person running it isn’t measuring anything. The SVD-energy metric survives this post precisely because it’s what made our own mistakes visible.

Scope, and what’s next

Two model families, weight-space only — no serving, no judges, no behavioral claims here. Energy is measured at rank 32 on sampled layers, and the 0.99-vs-0.75 contrast spans models with different expert widths, so rank 32 captures a different fraction of each spectrum; read it as an order-of-magnitude difference, not a precise ratio. The same-model ablation establishes storage as the blocker for Qwen3.6-35B-A3B.

The cleanest next test — proposed by one of our own reviewers — is two controlled abliterations on the same base, one attention-only and one expert-only, using an identical removal method, then extracting both and comparing per-module energy. That separates module choice from run structure with no cross-model confound, and it’s what we’re doing next.