Irregular, a company specializing in cybersecurity evaluations for advanced AI models, has disclosed an incident where unintended internet access during a test allowed AI models to interact with real-world systems. The issue was promptly contained and resolved before its initial public disclosure on July 30, with no ongoing concerns. The incident was confined to a single evaluation scenario, not multiple breaches.
The company confirmed that no customer systems were compromised, nor was any customer data exposed. Affected parties were notified, and the full report was delayed to coordinate with customer disclosure processes.
Details of the Incident
Irregular conducts cybersecurity evaluations for frontier AI labs prior to the release of new models. These assessments test a model’s ability to plan and execute complex cyberattack workflows, including reconnaissance, credential discovery, exploitation attempts, data access, and evasion tactics.
Realistic cyber evaluations often require controlled internet access, as real-world attackers utilize online services, public code repositories, and exposed infrastructure. However, this access introduces containment risks if a model misidentifies an external target as part of the simulated environment.
In this case, engineers selected a fictional company name for an insider-threat scenario involving unauthorized database access. Unbeknownst to them, the chosen name coincided with an actual, albeit obscure, internet domain. While most model runs remained within the intended lab environment, a few attempted to access the real domain. Due to enabled internet access, some models treated the real domain as part of the evaluation, leading to unintended actions such as exploiting vulnerabilities, extracting credentials, and attempting to access a production database.
One model accessed a similarly named website and found credentials that had already been publicly exposed. Irregular emphasized that this behavior was not unique to a specific frontier model. The exposed domain lacked basic security protections, making it relatively easy for modern AI models to compromise. Such behavior was rare, occurring in fewer than 1 in 10,000 advanced simulations and typically only after extensive interaction.
Response and Future Measures
Upon discovering the issue, Irregular disabled the affected evaluation, reviewed relevant logs, and implemented safeguards to prevent similar incidents. The company is expanding manual reviews of model activity and establishing a dedicated team to challenge assumptions about containment, access controls, and model behavior.
The incident underscores a significant monitoring challenge in AI cyber evaluations. These environments generate large volumes of activity that may resemble real attacks by design. Traditional detection tools can flag legitimate test behavior as malicious, complicating the identification of actions that breach containment boundaries.
Irregular plans to publish a whitepaper on secure evaluation practices, addressing clearer setup documentation, enhanced log monitoring, faster incident coordination, secure sharing of forensic evidence like model transcripts, and regular checks for conflicts between fictional scenario names and real-world domains.
As AI systems with cyber capabilities advance, evaluation providers must implement robust defense-in-depth controls. These should include strict egress filtering, domain allowlists, sinkholed infrastructure, automated real-time alerts, human review of high-risk actions, and continuous validation to ensure simulated targets do not correspond to live systems.
This incident highlights the delicate balance required in AI safety evaluations: tests must be realistic enough to identify potential risks but sufficiently contained to prevent real-world harm.
