The U.S. Cybersecurity and Infrastructure Security Agency (CISA), joined by the NSA and FBI, has issued a serious warning: several China-based AI firms are accused of extracting responses from U.S. advanced AI models at massive scale. Their goal—creating synthetic datasets that replicate key capabilities without permission.
What CISA Uncovered
According to a new advisory, starting in late 2024 and continuing through mid-2025, certain Chinese AI companies—DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI—are alleged to have harvested billions of tokens via millions of exchanges with models like Claude, GPT, Gemini, and Grok. The data extracted covered reasoning, programming, software engineering, mathematical tasks, legal functions, and even “character creation.”
This behavior goes beyond what’s common in AI research. CISA calls it a form of industrial-scale knowledge distillation: aggregating large amounts of model output to train competing systems without authorization. The concern is not malware or traditional hacking—it’s essentially cloning performance by replaying a model’s internal skills.
Methods & Risks Identified
The firms allegedly used techniques like API proxies, cloud services, aggregators, and “transfer stations” to hide user metadata and evade geo-restrictions, making it harder for providers to pinpoint where requests came from. Bulk subscriptions and multiple accounts, often shifting among providers, were part of the strategy. Prompt injection and jailbreak-style queries also reportedly forced models to leak internal reasoning steps.
CISA emphasized that this type of extraction poses national security and economic risks by cutting down the time and expense needed to develop competitive models. Also troubling: the alleged activities seem to have occurred with the knowledge—or at least the tolerance—of the Chinese government, though no direct proof of government control was cited.
Recommendations for AI Providers
To combat these extraction campaigns, CISA advises providers to tighten identity verification, watch for high subscription-to-usage ratios, impose rate limits, and log requests across systems. Higher-confidence malicious requests could receive lower fidelity responses or altered behavior to make extracted data less useful. Techniques like differential privacy, adversarial testing, prompt-injection hardening, and stricter API controls are also suggested.
No specific indicators of compromise—such as malicious IPs or domains—were shared. That makes detection trickier, especially since the threat actors are careful to avoid leaving obvious traces.
This isn’t the first time model output collection has raised alarm. Prior reporting has documented unauthorized extraction from models like Claude. The latest advisory confirms that such practices are more widespread and systematically organized than previously known.
Analyzing this advisory, it seems the AI ecosystem is at a tipping point: protecting model functionality is now as critical as securing data. Providers must invest heavily in defense around model output, API access, and behavior monitoring. Going forward, we’ll need transparency about downstream use of extracted data and possibly new industry norms or regulations to prevent imitation via stealth collection. What companies do next might define who leads the next wave of frontier AI.
