In February 2026, Anthropic publicly accused three Chinese AI labs (DeepSeek, Moonshot AI, and MiniMax) of generating more than 16 million interactions with Claude through approximately 24,000 fraudulent accounts, with the explicit goal of siphoning off its capabilities in reasoning, tool usage, and code generation to train their own competing models. Four months later, in June 2026, the company filed a similar charge—and even broader—against operators linked to Alibaba: 28.8 million interactions via 25,000 fraudulent accounts. These two cases, documented by Anthropic in a public statement and a letter to the U.S. Senate, remain allegations at this stage, but they illustrate on an unprecedented scale an attack vector that requires neither hacking nor privileged access: model extraction.
What is the knowledge distillation model?
Model extraction involves reconstructing a functional approximation of a proprietary model by making a large number of queries through its API, then using the responses obtained to train a second model that replicates its behavior. Technically, the dominant method is called knowledge distillation: rather than learning from raw data, the “student” model is trained directly on the responses of the “teacher” model, allowing it to capture refined capabilities (reasoning, style, use of tools) without ever having access to the original model’s architecture or weights.
On the surface, this mechanism is nothing new; it was formalized as early as 2016 for traditional classification models, but its application to large language models changes its scope: distillation requires no knowledge of the target architecture, runs using standard fine-tuning tools, and, according to documented cases, can produce a student model whose performance comes very close to that of the original model on the targeted tasks.
Model Extraction vs. Model Poisoning: What’s the Difference?
Model extraction targets a model once it is in production, with the goal of cloning it; this is the opposite of model poisoning, which seeks to corrupt a model during training in order to implant hidden behavior into it. The two approaches also differ in terms of the harm they cause: poisoning compromises a model’s integrity and reliability, while extraction compromises the intellectual property and competitive advantage represented by a model that has been trained or fine-tuned at great expense.
How can you detect an AI model extraction attack?
An extraction attack differs from legitimate use in its characteristic query patterns, which the most recent defenses seek to detect: a volume of queries far exceeding normal usage, systematic coverage of the space of possible inputs rather than scattered, natural questions, and sequential variations suggesting methodical exploration rather than human use. In thecase uncovered by Anthropic, one signal was particularly revealing: one of the targeted labs conducted more than 150,000 exchanges specifically designed to guide Claude through its reasoning step by step, thereby producing directly reusable reasoning chain training data—a query pattern largely incompatible with ordinary conversational use.
Best Practices for Protecting an LLM Model Against Extraction
- Limit the information included in each response: Return only the final prediction (top-1 / generated text) and avoid detailed confidence scores or complete probability distributions, unless explicitly required for functional purposes. Truncate or round off scores and, if necessary, add calibrated noise to logprobs to reduce the precision that can be exploited by distillation. These measures increase the number of queries required and degrade the quality of the stolen model.
- Monitor query patterns rather than each query individually: Analyze traffic by time window and by identity (API key, account, IP) to detect data extraction patterns (sustained volume, high diversity of prompts, repetitive templates, systematic coverage of the input space). Implement anomaly detection that scores volume, the ratio of unique inputs, and the proportion of information-rich requests. This approach makes it possible to identify campaigns even when each individual request appears benign
- Limit throughput and volume per account: Apply multi-layered rate limiting (by API key, by IP address, daily/monthly quotas) scaled according to legitimate use cases. In the event of repeated threshold exceedances, use progressive throttling (back-off, CAPTCHA, temporary suspension). While these measures will not stop a sophisticated distributed attack, they significantly increase its cost and visibility.
- Consider marking the model’s behavior: Embed a statistical watermark in the outputs using schemes designed to resist distillation (patterns injected into the probabilities based on a secret key). Establish a post-hoc verification process by testing a suspicious model with dedicated queries to detect the presence of the watermark. Watermarking does not prevent extraction, but it provides evidence of extraction to protect intellectual property or enforce terms of use.
FAQ: Everything You Need to Know About AI Model Theft and Extraction
Does the model extraction process apply only to large AI labs?
No. Any company that has fine-tuned a model using high-value proprietary data (industry knowledge, specific business terminology, accumulated expertise) is at risk as soon as that model is exposed via an API, regardless of the company’s size.
Is a standard flow limit sufficient to prevent extraction?
It slows it down without completely stopping it, particularly in the case of an attack spread across many accounts, such as the one documented by Anthropic. It remains, however, a basic measure that should be combined with monitoring of query patterns.
How can one prove that a competitor's design was copied from one's own?
This is a real challenge, illustrated by the fact that the allegations documented here are based on an analysis of the original model’s query logs rather than on direct evidence from the copied model. Behavioral tagging techniques are specifically designed to facilitate this demonstration should it become necessary.
Protect Your LLMs and Fine-Tuned Models from Extraction
Our experts assess the exposure of your proprietary models and set up monitoring for unusual query patterns.
Would you like to discuss your project or assess your needs? Contact our experts.