Defeating Prompt Injections by Design
Authors: Edoardo Debenedetti, Ilia Shumailov, Tianqi Fan, Jamie Hayes, Nicholas Carlini, Daniel Fabian, Christoph Kern, Chongyang Shi, Andreas Terzis, Florian Tramèr
Published March 24, 2025
Read the paper: arXiv 2503.18813 · PDF
Why it matters
Most prompt injection defenses ask the model to resist instructions hidden in the data it reads. CaMeL starts from the assumption that the model may still be fooled and puts the protection in the system around it. It turns the user's request into a program before any untrusted data is read, so an email or web page the agent fetches later can't add steps or change the order of steps.
It also tags values with capabilities that record where the data came from and who may receive it. Before a tool runs, a policy checks those tags, which is how it stops private data from flowing somewhere the user never allowed. The cost the authors report is real but modest: 77% of AgentDojo tasks solved with the defense on, against 84% for the same agent with no defense.
How to use it
Apply it to any agent with tool use that reads untrusted content. Decide the plan from the user's request before you fetch outside data. Treat whatever comes back from a tool as data that can fill in values, never as something that can choose the next tool. Write explicit policies for the tools that send, share or delete, and check them in code at call time. Keep your guardrails as an additional layer, since the paper's guarantee covers the data flows CaMeL tracks and depends on the policies you write.
Abstract
Large Language Models (LLMs) are increasingly deployed in agentic systems that interact with an untrusted environment. However, LLM agents are vulnerable to prompt injection attacks when handling untrusted data. In this paper we propose CaMeL, a robust defense that creates a protective system layer around the LLM, securing it even when underlying models are susceptible to attacks. To operate, CaMeL explicitly extracts the control and data flows from the (trusted) query; therefore, the untrusted data retrieved by the LLM can never impact the program flow. To further improve security, CaMeL uses a notion of a capability to prevent the exfiltration of private data over unauthorized data flows by enforcing security policies when tools are called. We demonstrate effectiveness of CaMeL by solving 77% of tasks with provable security (compared to 84% with an undefended system) in AgentDojo. We release CaMeL at https://github.com/google-research/camel-prompt-injection.
Articles that use this
- Red Teaming Your AI Agents With Prompt Chains Before Attackers DoSecurity
- Prompt Patterns That Catch MCP Tool Poisoning Before Your Agent Executes ItSecurity
- Red-Teaming Your Own AI App: A Prompt Injection Test Suite You Can Run TodaySecurity
- Building Prompt Guardrails: Input Filtering and Output Validation for LLM-Powered AppsSecurity
- Prompt Injection Testing for Production Apps: A Red Team PlaybookSecurity

