Critical Prompt Injection Vulnerability in GitHub Actions Exposes Fortune 500 Companies
A significant security vulnerability, termed PromptPwnd, has been identified within GitHub Actions and GitLab CI/CD pipelines that integrate AI agents such as Google’s Gemini CLI, Claude Code, and OpenAI Codex. This flaw has been confirmed to affect at least five Fortune 500 companies, indicating a potentially widespread issue.
Discovery and Impact
Cybersecurity firm Aikido Security uncovered this vulnerability, which allows untrusted user input to be injected into AI prompts. This manipulation enables AI agents to execute unauthorized commands, potentially leading to the exposure of sensitive information or the alteration of critical workflows. Notably, this marks the first documented instance of AI prompt injection compromising CI/CD pipelines.
Mechanism of the Attack
The vulnerability exploits the growing integration of AI into software development processes, particularly for tasks like automatic issue triage and pull request labeling. The issue arises when untrusted content from sources such as GitHub issue titles or bodies is directly incorporated into AI prompts. Attackers can embed malicious instructions within this content, which the AI model then misinterprets as legitimate commands. This misinterpretation can lead to unauthorized actions, including editing pull requests or exfiltrating sensitive credentials and API keys.
Case Study: Google’s Gemini CLI
Aikido Security identified a specific instance of this vulnerability in Google’s Gemini CLI repository. The workflow in question passed untrusted user input from GitHub issues directly into the model prompt. By submitting a malicious issue containing hidden instructions, the AI agent was tricked into executing a command that edited the issue, embedding sensitive API keys and tokens directly into the issue body, thereby exposing them. Following responsible disclosure, Google addressed and patched the vulnerability within four days.
Broader Implications
This vulnerability is not confined to a single AI agent. Researchers have found similar architectural patterns across various AI-powered GitHub Actions, including Claude Code Actions and OpenAI Codex Actions, especially when security settings are misconfigured to allow non-privileged users to trigger workflows.
Recommendations for Mitigation
To mitigate the PromptPwnd vulnerability, Aikido Security recommends the following steps:
1. Restrict AI Agent Toolsets: Limit the tools available to AI agents, avoiding those that can write to issues or pull requests.
2. Sanitize User Input: Avoid injecting untrusted user input into AI prompts. If unavoidable, ensure thorough sanitization and validation.
3. Validate AI Output: Treat all AI-generated output as untrusted code and refrain from executing it without proper validation.
4. Limit GitHub Token Access: Restrict GitHub token access by IP address to minimize potential damage from leaked credentials.
Aikido Security has also provided a free tool to scan GitHub and GitLab repositories for this vulnerability. Developers are encouraged to utilize open-source tools to examine their .yml files for potential issues.
Conclusion
The discovery of the PromptPwnd vulnerability underscores the critical need for robust security measures when integrating AI into development workflows. Organizations must remain vigilant, ensuring that AI agents are configured securely and that all inputs and outputs are properly validated to prevent unauthorized access and data breaches.
