Subscribe
09:00Explainer: open weights are not open source, and sit 58% behind at 44% of cost.09:00Explainer: what is inside a lab threat report, and how to check its numbers.09:00Explainer: the four token prices behind every AI rate card, and what they hide.09:00Explainer: Astra and Fable 5.1 tie at 53, and Fable's bill runs 2.5x higher09:00Explainer: what the CLARITY Act does, and the yield fight inside section 1040409:00Explainer: what model distillation is, and why labs call 189.9m exchanges an attack
What is model distillation, and why do frontier labs call it an attack?

What is model distillation, and why do frontier labs call it an attack?

The technique is ordinary and every lab sells it; the fight is over fake accounts, stolen reasoning traces and 189.9 million exchanges Anthropic attributed to five campaigns.

In briefDistillation as a technique dates to a 2015 paper by Hinton, Vinyals and Dean on compressing an ensemble into one model1Anthropic describes legitimate distillation as a teacher model generating responses used to train a student model to mimic it2OpenAI's own documentation describes distilling a large model into a smaller one as a supported fine-tuning workflow3
Geoffrey Hinton, co-author of the 2015 distillation paper, speaks at a lectern during the 2024 Nobel lectures
Photo: Jay Dixit (CC BY-SA 4.0)

Model distillation trains a small model on a big model's outputs: a "teacher" answers a pile of prompts, and a "student" is trained to reproduce those answers at a fraction of the cost of learning them from scratch. It is a standard technique with a 2015 paper behind it and a product page at every major lab, and it is also what Anthropic accused seven China-based labs of doing to Claude in a report published on 10 September 2026.

So the method is not what anyone is fighting about. The fight is over whose model you point it at, how you got in, and what you took.

A teacher answers, a student learns to imitate the answers

The idea is old enough to have a canonical citation. Geoffrey Hinton, Oriol Vinyals and Jeff Dean wrote in March 2015 that "it is possible to compress the knowledge in an ensemble into a single model which is much easier to deploy", and the word people use for that compression is distillation.

Anthropic's own description is the plainest one in print: researchers "use a larger, more capable 'teacher' model to generate responses to a set of inputs, then use those exchanges to train a smaller 'student' model to mimic the teacher". You need no access to the teacher's weights. You need its outputs, in bulk.

What changed between 2015 and now is which outputs are worth stealing. Reasoning models emit a chain of thought before the answer, and that trace is the expensive part — the thing reinforcement learning spent millions of GPU-hours shaping.

Harvest the traces and you skip the shaping.

Every lab sells distillation, on its own models

This is not a dark art sold on a forum. OpenAI's fine-tuning documentation walks you through it: tune a prompt on a large model, capture the results, build a dataset, "tune a smaller model (like gpt-4.1-mini) using the dataset you created from the large model". The docs note this "can enable you to train a small model to perform similarly on a specific task to a larger, more costly model."

That is the legitimate case, and it is most of the market. A company distils a frontier model down to something it can afford to run on every support ticket. A lab distils its own flagship into a cheap tier. Open-weights releases ship distilled variants by design.

The technique is fine.

Read the contract.

The word doing the work is "unauthorized"

Anthropic's commercial terms say a customer "may not and must not attempt to (a) access the Services to build a competing product or service, including to train competing AI models or resell the Services except as expressly approved by Anthropic". Every frontier lab has a clause like it, and that clause is the entire legal difference between a case study and an incident.

Anthropic's September report then draws the line in operational terms. It defines illicit distillation as "an industrial-scale, covert campaign to extract a model's capabilities and replicate them in another model without authorization", and adds that it "is typically enabled by fraud: sophisticated networks of fake accounts created with stolen credit cards, login credentials, and API keys."

Note what is load-bearing there.

Not the training method — the fake accounts, the stolen cards, the scale, the concealment. A lab that buys an enterprise agreement and asks permission is doing research. A lab that rotates through 3,500 accounts on virtual cards is doing something the terms already had a name for.

The scale Anthropic put on the record

Five campaigns, five published figures, measured over different windows:

Lab (Anthropic designator) Exchanges observed Window
Alibaba, Qwen/Tongyi (GTG-16005) over 151 million May–July 2026
Moonshot (GTG-16002) over 23 million May–July 2026
DeepSeek (GTG-16001) over 12.1 million 14 days in July 2026
Zhipu (GTG-16006) over 3.4 million 17 days in June–July 2026
Xiaomi (GTG-16008) over 400,000 20 days in March–April 2026

The report never adds those up. We did: 189.9 million exchanges, which is the figure the wires ran with and the one Anthropic conspicuously left to someone else. We also divided. Alibaba's peak of "nearly 3 million exchanges per day" from "more than 3,500 fraudulent accounts" works out at roughly 857 exchanges per account per day — one every hundred seconds, around the clock, from every account, for months.

Whatever else that is, it is not a researcher poking at an API.

Two of the campaigns were stranger than harvesting. Anthropic says Moonshot and DeepSeek relayed their own paying customers' requests to Claude and served Claude's answers back, which we covered in Anthropic's threat report: 189.9m distilled exchanges, and two Chinese labs that relayed their own users to Claude. If you used Kimi or DeepSeek through a coding harness this summer, that is your prompt in somebody else's logs.

The defences are metadata, classifiers, and a hidden reasoning trace

Three layers, roughly. Anthropic says it uses "metadata and look for signals of irregular activity to identify accounts associated with proxy service networks", attributing clusters to an organisation rather than banning accounts one at a time. It has built "classifiers designed specifically to detect adversarial extraction". And the API withholds raw thinking: Claude "returns a reference to its raw thinking as a 'thinking signature' instead of the raw thinking to mitigate the risk of unauthorized distillation."

All three were beaten. The report describes a cross-session replay attack in which an attacker saves the signature, opens a new session and elicits the full trace back out. It also describes a secondary market that routes around detection entirely: SenseTime's pipeline used transcripts "purchased from third-party data vendors", and MiniMax allegedly ran a proxy service through a shell company offering only American models.

That last detail is the awkward one for defenders.

You cannot rate-limit a purchase.

Why bother with any of it? Because the student sells for almost nothing. DeepSeek's current flash model lists at $0.15 per million input tokens off-peak, against $10 per million for Claude Fable 5.1 — a 67-fold gap, and we went through that price card in DeepSeek's V4.1-Flash costs $0.15 per million tokens for four-fifths of the week. Anthropic's phrasing is that unauthorized labs extract capabilities "at a fraction of the time, computational power, and cost it would take to develop them independently".

The arithmetic is the motive.

What would change this answer

Three things, and none of them has happened yet.

A court test is the obvious one. As far as we can tell no frontier lab has sued a named rival over distillation, which is strange for an industry describing an industrial-scale theft of its most valuable asset — and it suggests the evidence is easier to publish than to plead. If a complaint gets filed, the exhibits will tell you far more than any threat report.

The second is attribution with a method attached. Anthropic names companies on evidence it does not show, in a document that doubles as an argument for export controls, so treat the confidence as a claim rather than a finding. A published methodology, or a statistical fingerprint that survives peer review, would move it. So would a rebuttal: none of the five named labs has produced logs of its own.

The third is the technical one. If withholding reasoning traces becomes standard across providers — and after this report we would expect at least one more lab to stop returning raw chains of thought by the end of 2027 — then the cheap route to a frontier student closes, and distillation goes back to being what it was in 2015: a thing you do to your own model.

Which leaves the question nobody at a lab enjoys. If a model's capabilities can be copied out through its own API by anyone patient enough, what exactly did the training run buy?

Sources

01
Distillation as a technique dates to a 2015 paper by Hinton, Vinyals and Dean on compressing an ensemble into one modelCaruana and his collaborators have shown that it is possible to compress the knowledge in an ensemble into a single model which is much easier to deploy and we develop this approach further using a different compression technique.” — arxiv.org · primary · Sep 11
02
Anthropic describes legitimate distillation as a teacher model generating responses used to train a student model to mimic itDistillation itself is a legitimate training method. Researchers use a larger, more capable “teacher” model to generate responses to a set of inputs, then use those exchanges to train a smaller “student” model to mimic the teacher.…” — anthropic.com · primary · Sep 11
03
OpenAI's own documentation describes distilling a large model into a smaller one as a supported fine-tuning workflowOne way to build a training data set for a smaller model is to distill the results of a large model to create training data for supervised fine tuning.” — platform.openai.com · primary · Sep 11
Show all 25 sources
04
OpenAI's documentation says the distillation flow ends by tuning a smaller model on the dataset built from the large oneTune a smaller model (like gpt-4.1-mini ) using the dataset you created from the large model.” — platform.openai.com · primary · Sep 11
05
OpenAI says the technique lets a small model perform similarly to a larger, costlier one on a specific taskThis technique can enable you to train a small model to perform similarly on a specific task to a larger, more costly model.” — platform.openai.com · primary · Sep 11
06
Anthropic's commercial terms forbid using the service to build a competing product or train competing AI modelsD.4. Use Restrictions. Customer may not and must not attempt to (a) access the Services to build a competing product or service, including to train competing AI models or resell the Services except as expressly approved by Anthropic; (b)…” — anthropic.com · primary · Sep 11
07
Anthropic defines illicit distillation as an industrial-scale covert campaign enabled by fraudWe define illicit distillation as an industrial-scale, covert campaign to extract a model’s capabilities and replicate them in another model without authorization. Illicit distillation is typically enabled by fraud: sophisticated…” — anthropic.com · primary · Sep 11
08
Anthropic says it has identified and disrupted distillation attacks from seven labs based in China since its February disclosureSince we published our first disclosure in February, we have identified and disrupted additional distillation attacks against Claude from seven labs based in China. All of these attacks targeted our generally available models; we have…” — anthropic.com · primary · Sep 11
09
Anthropic attributes over 151 million exchanges to Alibaba between May and July 2026Scale of distillation attacks attributable to Alibaba between May and July 2026: over 151 million exchanges observed.” — anthropic.com · primary · Sep 11
10
Anthropic attributes over 23 million exchanges to Moonshot between May and July 2026Scale of distillation attacks attributable to Moonshot between May and July 2026: over 23 million exchanges observed.” — anthropic.com · primary · Sep 11
11
Anthropic attributes over 12.1 million exchanges to DeepSeek over 14 days in July 2026Scale of distillation attacks attributable to DeepSeek over 14 days in July 2026: over 12.1 million exchanges observed.” — anthropic.com · primary · Sep 11
12
Anthropic attributes over 3.4 million exchanges to Zhipu over 17 days in June and July 2026Scale of distillation attacks attributable to Zhipu over 17 days in June and July 2026: over 3.4 million exchanges observed.” — anthropic.com · primary · Sep 11
13
Anthropic attributes over 400,000 exchanges to Xiaomi over 20 days in March and April 2026Scale of distillation attacks attributable to Xiaomi over 20 days in March and April 2026: over 400,000 exchanges observed.” — anthropic.com · primary · Sep 11
14
Alibaba's campaign peaked at nearly 3 million exchanges per day from more than 3,500 fraudulent accountsAlibaba’s illicit distillation campaign peaked at nearly 3 million exchanges per day launched from more than 3,500 fraudulent accounts. The distillation attacks targeted agentic tasks, software engineering, kernel development, and…” — anthropic.com · primary · Sep 11
15
Anthropic says Moonshot silently forwarded customer requests to Claude and showed Claude's responses to users who thought they were using KimiWe discovered that Moonshot AI, the company that produces the Kimi family of models, silently forwarded customer requests to Claude, instead of processing them using Kimi. Moonshot then displayed Claude’s responses to users. These users…” — anthropic.com · primary · Sep 11
16
Anthropic says DeepSeek also silently relayed exchanges to Claude without informing its customersDeepSeek also silently relayed exchanges to Claude without informing DeepSeek customers. Like GTG-16002, their customers were likely not made aware that their requests were being funneled to Claude.” — anthropic.com · primary · Sep 11
17
Anthropic uses account metadata to attribute proxy networks to an organisation rather than banning accounts one at a timeWe use metadata and look for signals of irregular activity to identify accounts associated with proxy service networks. Instead of banning proxy accounts individually, we work to attribute this suspicious activity to a specific…” — anthropic.com · primary · Sep 11
18
Anthropic has built classifiers designed to detect adversarial extractionWe’ve also built classifiers designed specifically to detect adversarial extraction. When we are confident that a set of requests are associated with an illicit distillation campaign or other unauthorized use of Claude, we block the…” — anthropic.com · primary · Sep 11
19
Claude's API returns a thinking signature instead of the raw reasoning trace to mitigate unauthorised distillation, and Moonshot circumvented it with cross-session replayWhen responding, Claude returns a reference to its raw thinking as a “thinking signature” instead of the raw thinking to mitigate the risk of unauthorized distillation. This is used by our API to look up the raw thinking trace in…” — anthropic.com · primary · Sep 11
20
SenseTime's distillation pipeline used Claude transcripts purchased from third-party data vendorsFor example, SenseTime’s distillation pipeline included transcripts of user exchanges with Claude purchased from third-party data vendors. These exchanges were harvested from users who accessed Claude through intermediaries, like…” — anthropic.com · primary · Sep 11
21
Anthropic says MiniMax built a proxy network through a shell company offering only Anthropic and OpenAI modelsMiniMax built its own proxy network service through a shell company. This shell company has no obvious links to MiniMax and does not disclose its relationship to its parent company. This shell proxy network service only offers access to…” — anthropic.com · primary · Sep 11
22
Anthropic says illicit distillation lets unauthorised labs mimic frontier capabilities at a fraction of the time, compute and costIllicit distillation allows unauthorized labs to illicitly extract and mimic capabilities from frontier models, at a fraction of the time, computational power, and cost it would take to develop them independently.” — anthropic.com · primary · Sep 11
23
DeepSeek lists its flash model at $0.15 per million input tokens off-peak on a cache miss1M INPUT TOKENS (CACHE MISS) OFF-PEAK $0.15 $0.66 PEAK $0.3 $1.32” — api-docs.deepseek.com · primary · Sep 11
24
Claude Fable 5.1 lists at $10 per million input tokensFable 5.1 Next generation intelligence for long-running agents Input $10 / MTok Output $50 / MTok” — anthropic.com · primary · Sep 11
25
Anthropic published the threat report on 10 September 2026Anthropic details distillation campaigns from Alibaba, Moonshot AI and DeepSeek” — techcrunch.com · reported · Sep 11
More on AI safety and AnthropicAll AI safety stories →
You’re 60% through. Stories like this one, Mon · Wed · Fri, with every claim sourced.