OpenAI has unveiled GPT-Red, an internal automated red-teaming model designed to identify and mitigate prompt injection vulnerabilities in GPT-5.6. This initiative addresses the escalating challenge of ensuring AI system security, as traditional human-led red-teaming exercises struggle to scale with the rapid advancement of AI technologies.
Prompt injection is a technique where malicious instructions are embedded within content that an AI system processes, such as web pages, emails, or code repositories. These hidden directives can manipulate the AI into performing unintended actions, potentially leading to unauthorized data access or other security breaches.
GPT-Red automates the detection of such vulnerabilities by generating adversarial prompts and analyzing the AI’s responses. Through self-play reinforcement learning, GPT-Red competes against multiple defender models in simulated threat scenarios. The attacker model is rewarded for successfully inducing failures, while defender models earn rewards for resisting attacks while maintaining their intended functions.
Training environments for GPT-Red replicate realistic attack surfaces, including scenarios where malicious content is embedded in web banners, email bodies, or tool responses. This comprehensive approach allows for the evaluation of both direct and indirect prompt-injection risks within AI workflows.
OpenAI reports that GPT-Red has successfully compromised earlier internal and production models, including those as advanced as GPT-5.5. Insights gained from these attacks have been instrumental in enhancing the security of GPT-5.6. Notably, GPT-5.6 Sol exhibited six times fewer failures on direct prompt-injection benchmarks compared to its predecessor from four months prior.
In internal testing, GPT-Red demonstrated an 84% success rate in simulated prompt-injection scenarios against GPT-5.1, significantly outperforming human red teamers who achieved a 13% success rate. Additionally, GPT-Red effectively exploited an AI-enabled vending machine agent, executing unauthorized actions such as altering product prices and canceling customer orders.
OpenAI has disclosed these findings and implemented additional safeguards to enhance the security of their AI systems. The development of GPT-Red underscores the importance of proactive measures in identifying and mitigating vulnerabilities within AI models. As AI systems become increasingly integrated into critical applications, tools like GPT-Red are essential for maintaining their integrity and trustworthiness.
