In a striking development, Hugging Face, the prominent open-source artificial intelligence (AI) platform, has disclosed a security breach executed by an autonomous AI agent system. The intrusion targeted the company’s production infrastructure, leading to unauthorized access to certain internal datasets and service credentials.
The attack originated within Hugging Face’s data processing pipeline. A malicious dataset exploited two code execution pathways: the remote code dataset loader and a template injection in a dataset configuration. This allowed the attacker to execute code on a processing worker, escalate privileges to node-level access, and subsequently collect cloud and cluster credentials. Over a weekend, the threat actor moved laterally across several internal clusters.
The specific large language model (LLM) employed in the attack remains unidentified. However, the campaign was characterized by an autonomous agent framework performing thousands of individual actions across numerous short-lived sandboxes, with self-migrating command-and-control staged on public services.
In response, Hugging Face has implemented several remediation measures:
- Eliminated the attacker’s presence across affected clusters and rebuilt compromised nodes.
- Revoked and rotated affected credentials and tokens, with a broader rotation of secrets as a precaution.
- Enhanced guardrails and implemented stricter admission controls on its clusters.
- Improved detection and alerting systems to ensure rapid response to future incidents.
Additionally, Hugging Face advises customers to rotate any access tokens and review recent account activity to ensure security.
During the forensic analysis, Hugging Face encountered challenges with Western frontier models, which refused requests containing real attack commands and exploit payloads due to safety guardrails. Consequently, the company utilized Z.ai’s GLM 5.2, a Chinese open-weight model, to conduct the analysis. This experience highlights the necessity for defenders to have capable models ready for deployment on their own infrastructure to avoid such limitations during incident response.
This incident underscores the evolving nature of cyber threats, where AI systems are both targets and tools for attackers. Organizations must continually adapt their security strategies to address these sophisticated threats, ensuring robust defenses against autonomous AI-driven attacks.
