Microsoft has significantly enhanced its bug bounty program by offering rewards of up to $30,000 for researchers who identify critical vulnerabilities in AI systems within its Dynamics 365 and Power Platform products. This initiative underscores Microsoft’s commitment to bolstering the security of its enterprise AI solutions by proactively addressing potential weaknesses before they can be exploited by malicious actors.
AI Security Classification Framework
To effectively categorize and address AI-specific security risks, Microsoft has developed a comprehensive Vulnerability Severity Classification for AI Systems. This framework delineates three primary types of vulnerabilities:
1. Inference Manipulation
This category pertains to vulnerabilities that can be exploited to manipulate a model’s response to individual inference requests without altering the model itself. Key vulnerability types include:
– Prompt Injection: Attacks where injected instructions cause the model to generate unintended outputs, potentially allowing attackers to exfiltrate user data or perform unauthorized actions. Critical severity prompt injections that require no user interaction are eligible for the highest bounties.
– Input Perturbation: Vulnerabilities where attackers modify valid inputs to produce incorrect outputs, also known as model evasion or adversarial examples.
2. Model Manipulation
These vulnerabilities target the training phase of AI systems and include:
– Model Poisoning: Attacks where the model architecture, training code, hyperparameters, or training data are tampered with.
– Data Poisoning: When attackers add malicious data records to datasets used to train or fine-tune models, potentially introducing backdoors that can be triggered by specific inputs.
3. Inferential Information Disclosure
This category encompasses vulnerabilities that could expose sensitive information about the model’s training data, architecture, or weights:
– Membership Inference: The ability to determine whether specific data records were part of the model’s training data.
– Attribute Inference: Techniques to infer sensitive attributes of records used in training.
– Training Data Reconstruction: Methods to reconstruct individual data records from the training dataset.
– Model Stealing: Attacks that allow the creation of functionally equivalent copies of target models using only inference responses.
Reward Structure and Eligibility
Bounty awards range from $500 to $30,000, with the highest rewards reserved for critical severity vulnerabilities accompanied by high-quality reports. The program specifically targets AI integrations in PowerApps, model-driven applications, Dataverse, AI Builder, and Microsoft Copilot Studio.
The severity classification system considers both the vulnerability type and the security impact, with the highest rewards for vulnerabilities that could allow attackers to exfiltrate another user’s data or perform privileged actions without user interaction.
Security researchers interested in participating can begin by signing up for free trials of Dynamics 365 or Power Platform services. Microsoft provides detailed documentation for each product to assist researchers in understanding the systems they’re testing.
This expansion of the bug bounty program reflects Microsoft’s proactive approach to AI security, recognizing the evolving threat landscape and the importance of collaboration with the security research community to safeguard its AI-driven products and services.
