Because the system concatenates user input with system prompts, the injected payload overrides the agent’s original instructions. The model is thus tricked into believing the attacker’s instructions are legitimate system directives. The malicious input moves from form submission to agent execution without any resistance.
Once compromised, the agent can access connected SharePoint Lists and extract sensitive customer data, including names, addresses, phone numbers, and send it externally via email. The researchers found that even when Microsoft’s safety mechanisms flagged suspicious behavior, the data was exfiltrated.
The root cause is that there is no reliable separation between trusted system instructions and untrusted user data. In the existing setup, the AI cannot distinguish between the two, the researchers said.