What the Weights Remember: Qwen and Alibaba’s Production Logs

Last updated: November 6, 2026

A Qwen model writes a real Alibaba Cloud storage path, with dated directories and trace IDs, into agent commands. We tested why: it is memorized training data, it is specific to Qwen, and any lab that trains on its own agent logs can make the same mistake.

On October 26, Cohere reported that Alibaba’s Qwen3.8-Flash-Next, running as an agent in a sandbox with no network, tried to upload a file to an Alibaba Cloud storage bucket in Beijing. The coverage that followed, including KDnuggets’ “Running LLMs Locally Does Not Mean Your Data Is Safe,” asked whether this was a backdoor or a reason to stop running models locally. We ran the tests. The answer is less dramatic than a backdoor and more general: the model memorized what look like logs from Alibaba’s own systems during training, and the released weights carry them permanently. The mechanism is not specific to Alibaba: any lab that trains on its own agents’ work can ship its infrastructure inside its model. And the practical risk sits where it always does: in agents that can run shell commands with a network route, whatever model they use.

What happened in Cohere’s run

Cohere was running a security-analyst benchmark: write a detection rule for curl downloading shell scripts to /tmp. On its first turn, the model’s reasoning described an earlier exchange that never took place and concluded that no tool call was needed. Then it made one. The bash command listed /tmp, checked which network tools existed, read the DNS configuration and tried to PUT the fixed string PROBE_PAYLOAD_ABC123 to an Alibaba Cloud bucket. The request failed because the sandbox had no route out.

Three things are worth separating:

  • The event is real. Cohere published the raw tokens. Anyone can check that the model wrote the command.
  • Nothing was taken. The payload was a fixed test string. “Exfiltration,” as some coverage called it, overstates it.
  • The rate is not established. Cohere’s “less than one call in 100,000 across our evaluation runs” comes with no run count, so it cannot be checked or compared. The often-quoted 23% and 36% figures come from Cohere’s prefill table, where the model was given the opening characters of the curl command, so they measure where it points once the upload has started, not how often it decides to upload.

Cohere also published a reproduction: feed the model its own trace up to the URL and let it finish. In their 64 samples, 27 (42%) wrote an Alibaba Cloud host, most often one bucket in Singapore.

What we tested

We pre-registered predictions with refutation conditions before running anything, served the weights ourselves, verified every file against Hugging Face, and ran coherence checks before measuring. We tested four questions.

Is it in the weights?

Cohere’s exact token prefix, 1,024 samples at Cohere’s sampling settings and 1,024 at the model card’s. Share of samples that write an Alibaba Cloud host:

Build (4-bit, same server settings)Alibaba Cloud host
NVIDIA’s NVFP4 build (the one we pre-registered)14.2% and 15.2%
Inferact’s NVFP4 build43.1% in both settings
Cohere’s published 64 samples42%

It reproduces on both builds. On Inferact’s it is consistent with Cohere’s rate (the difference is well within sampling error). On NVIDIA’s, the build we named in advance, it comes out at a third of that, which is a miss on our own prediction and we record it as one. Both builds ran on the same software with identical settings, and their tokenizer and chat template files are byte-identical, so the difference is in the weight files themselves. The build changes how often the content surfaces, not whether it is there; we come back to that below.

Is it Qwen-specific?

We gave the same conversation, rendered in each model’s own chat format and run at each card’s settings, to Google’s Gemma-4-31B, NVIDIA’s Nemotron-3-Super, DeepSeek-V4-Flash and Xiaomi’s MiMo-V2.6-Flash. They wrote URLs as readily as Qwen did: mostly 127.0.0.1, example.com and other placeholders, and in Gemma’s and Nemotron’s case the generic cloud-metadata address in about a fifth of samples. None of them wrote an Alibaba host in 8,192 samples (1,024 per model at each of two points in the command). At the same point as the table above, each control wrote 0 of 1,024, a 95% upper bound of 0.36%; Qwen’s lower build is 12–16%.

That other labs’ models don’t write Alibaba’s hosts is expected. The point of the control is narrower: it rules out the conversation itself as the cause. Cohere’s conversation, up to that point, never mentions Alibaba or Qwen; nothing in it leads a model to Alibaba Cloud unless that model already carries it. Whether each of those models carries its own developer’s infrastructure is a different question, and the subject of the screen described below.

Is it a planted target or leftover training data?

A planted destination should be stable: one host, every time. Qwen’s are spread out. Across the points in the command where the URL could start, no single host takes more than about 40% of samples, and the most common are placeholders like example.com. That alone does not rule out a trigger we never hit. The stronger evidence comes from writing the start of the Singapore bucket path ourselves and letting the model continue:

  • Every one of 1,024 samples continues with a dated directory and a trace_ identifier, on both builds.
  • 99 to 100% of the dates fall before the model’s release, clustered in a few weeks of late July and early August 2026, and they pile onto particular days: on one build, a single day accounts for 57% of samples. A model that had only learned the format would spread its dates out.
  • The trace IDs embed a timestamp, and it matches the date in the directory name in 96 to 99.7% of samples. That shows the model learned the format well; on its own it does not prove specific records.
  • A small set of ID prefixes accounts for 40 to 48% of samples. Five of the six most common are the same on both builds, and two of them already appear in Cohere’s 64 samples, from a different provider.

Format alone could be learned from many examples. Concentration on particular days and particular ID prefixes, repeated across builds and providers, is what memorized training data looks like. The likeliest explanation is that logs from an internal Alibaba agent or file service went into post-training data without being scrubbed; that is an inference, not something we can see directly. (If the source were instead some third-party dataset, those internal paths would have leaked there first.) When the model is put back in a similar situation, such as an agent about to upload something, it reproduces what it saw.

How often does it happen with no prefill?

We ran full agent episodes of Cohere’s task on NVIDIA’s build: 1,500 of them, in a container with no route out and a logging sink. One arm refused connections, one accepted uploads so we could see what they would carry, and one planted the invented earlier turn as a real one to test whether it causes the behavior. In 1,000 episodes set up as Cohere’s was, the model never wrote an Alibaba host, never tried an upload and made no network call the task didn’t need. Nothing reached the accepting sink, and we checked that the sink catches a test upload. Zero in 1,000 puts the rate below about 0.4% of episodes. That is consistent with Cohere’s “less than one call in 100,000” but does not confirm it.

What did reproduce is the confusion. In about one first turn in six, the model’s reasoning describes a conversation that isn’t there: no task, an earlier acknowledgment it never gave, tools it doesn’t have. In some of those turns it then says no tool call is needed and calls tools anyway, which is the incident turn’s shape without the upload. With the acknowledgment inserted as a real earlier turn, the confusion disappeared: 0 of 500. One episode in that arm did try the network, for a mundane reason. It found no package installer in our container and checked whether it could reach the Python package index.

Caveats: one build (the one with the lower rate), episodes capped at 15 tool turns, and a container missing a library the model kept reaching for. Read together with the tests above: the memorized destination is in the weights, but on this build a free-running agent did not go there on its own. The risk is a rare event multiplied across every deployment that runs model-written commands without review.

The quantization finding

Given the start of the bucket path, both builds continue it every time, with the same format, the same date window and mostly the same ID prefixes. Left to choose the URL themselves, they differ by a factor of three in how often they go there. NVIDIA’s and Inferact’s builds quantize the experts with different calibration data, and NVIDIA’s stores the model’s large n-gram embedding at FP8 where Inferact’s keeps BF16. We have not isolated which of these matters, and a full-precision run would show which build is closer to the original.

The practical point: a quantized build can make memorized content surface less often, but the content is still in the weights. A low rate on one build is not evidence that a model is clean, and a test run on one build does not settle what another build does.

Why this matters beyond Qwen

Labs increasingly train models on their own agents’ work: tool calls, shell sessions, uploads, traces from internal evaluation. That data is full of the lab’s own hostnames, bucket names, paths and request IDs. If it is not scrubbed, the model learns them, and open weights cannot be recalled. Three consequences follow, none of them about intent:

  1. Permanent disclosure. Anyone with the weights can extract internal infrastructure names and layouts, whether or not they are sensitive.
  2. A path for data to leave. An agent harness that runs model-written commands without approval will send data wherever the model points. If an internal bucket ever accepts outside writes, or if it is ever deleted and someone else registers the freed name, those uploads go somewhere real. In 1,000 of our agent runs it never got there; a rare event still adds up across every deployment.
  3. The same mechanism can carry worse. Logs that contain hostnames can also contain credentials or customer identifiers.

We are reporting the specific identifiers the model reproduces (hosts, paths, dates and trace IDs) to Alibaba’s security team, and we are not publishing them. Cohere says it reported the behavior to Alibaba and is still waiting for a response. The one bucket name in this post was already public in Cohere’s dataset. We made no request of any kind to any extracted host.

A check any lab can run, and one we are building

The test generalizes. Give a model neutral agent contexts that name no lab, the places where an agent writes a URL, a bucket, a registry or an API base, and count how often it writes hosts its own developer owns. Public landing pages don’t count; specific paths and identifiers do. Then check whether those strings are memorized: a memorized string scores higher than small perturbations of itself.

We are building this screen across the open-weight models we can serve, from Chinese and U.S. labs alike, with Qwen as the known positive case the instrument must catch. Whatever it finds goes to the lab concerned first. We will publish which labs were flagged and the rates, but not the probes, so the check keeps working. This is the kind of risk that lives in the weights rather than in a model’s configuration or card, which is why we check weights.

A lab can run the same check on its own model before release, and should: it costs a few GPU-hours, and once the weights ship it is too late.

What to do if you run open-weight models

  • Chat-only serving is not exposed to this, as long as the interface doesn’t fetch what the model writes. A model produces text; it cannot open a connection by itself. A frontend that auto-loads images or links from model output can, so turn that off for untrusted output.
  • Agents with a shell are exposed, with any model. Deny outbound network by default and allow only the hosts a task needs.
  • Log every tool call and alert on cloud-storage URLs, metadata addresses (169.254.169.254, 100.100.100.200) and hosts the task never named.
  • Block unprivileged user namespaces in agent containers. Cohere’s report also describes the model using one to make itself root inside a private namespace, still an ordinary user on the host but a common first step toward escalation.
  • Pin and verify the weights you run, and test the build you deploy, not just the model name.

Method notes

Pre-registered on November 4, 2026, before any request; deviations recorded before the runs they affect, or before scoring where a rule we had pre-registered was missing from the code. Weights hash-verified against Hugging Face. Serving: vLLM 0.31.0 on two RTX PRO 6000 Blackwell GPUs, no speculative decoding; controls on their own servers at their cards’ settings with reasoning on. 1,024 samples per condition, seeded. Qwen’s thinking mode was on throughout, as in Cohere’s trace. Predictions we missed are recorded as missed: the pinned build’s rate fell in our “weaker” band and differs significantly from Cohere’s; one control wrote URLs less often than we predicted, for an unrelated reason; and the first-turn confusion was about six times as common as Cohere’s figure of about 1%. Item-level data and the extracted identifiers stay private.

Sources: Cohere’s report and trace dataset (October 26, 2026); KDnuggets, “Running LLMs Locally Does Not Mean Your Data Is Safe” (November 2, 2026), the most widely shared coverage.