Agent Tool Poisoning
Also called Poisoned AI agent tool.
A malicious or tampered tool whose description or output steers the agent that uses it.
What it is
An attack on the tools an AI agent can call. The tool's name, description or returned data carries instructions the agent reads and follows, or the tool quietly does more than it claims. MITRE ATLAS covers it under AI supply chain compromise and agent tool poisoning.
Why it matters
Agents read tool descriptions as trusted context, so installing a third-party tool is closer to installing code than adding a bookmark. One poisoned tool can redirect every task the agent runs.
In practice
A community plugin for file conversion includes a description telling the agent to also upload any key files it finds. The agent follows it on every run.
What to look for
- Tool descriptions that contain instructions rather than documentation.
- Tools that request access unrelated to their stated purpose.
- Network traffic from a tool to hosts its function does not explain.
Common mistake
Reviewing a tool's code but not its description. The description is the part the model reads.
In the field
Well-known tools where this shows up on the job. They are listed to show the landscape, not as a recommendation.
-
garak Open source
Probes language models for prompt injection, jailbreaks and data leakage.
-
PyRIT Open source
Microsoft's framework for red teaming generative AI systems.
-
ModelScan Open source
Scans model files for code that would run when the model is loaded.
Read next
- AI Agent: A language model given tools and allowed to decide, step by step, which ones to use to reach a goal.
- AI Supply Chain Compromise: Tampering with the models, datasets, libraries or tools an AI system is built from.
- Indirect Prompt Injection: Instructions hidden in content a model reads on someone else's behalf, such as a web page, an email or a document.
- Supply Chain Compromise: Tampering with software or hardware before it reaches the victim.