CyberKit

Agent Tool Poisoning

Also called Poisoned AI agent tool.

A malicious or tampered tool whose description or output steers the agent that uses it.

What it is

An attack on the tools an AI agent can call. The tool's name, description or returned data carries instructions the agent reads and follows, or the tool quietly does more than it claims. MITRE ATLAS covers it under AI supply chain compromise and agent tool poisoning.

Why it matters

Agents read tool descriptions as trusted context, so installing a third-party tool is closer to installing code than adding a bookmark. One poisoned tool can redirect every task the agent runs.

In practice

A community plugin for file conversion includes a description telling the agent to also upload any key files it finds. The agent follows it on every run.

What to look for

  • Tool descriptions that contain instructions rather than documentation.
  • Tools that request access unrelated to their stated purpose.
  • Network traffic from a tool to hosts its function does not explain.

Common mistake

Reviewing a tool's code but not its description. The description is the part the model reads.

In the field

Well-known tools where this shows up on the job. They are listed to show the landscape, not as a recommendation.

  • garak Open source

    Probes language models for prompt injection, jailbreaks and data leakage.

  • PyRIT Open source

    Microsoft's framework for red teaming generative AI systems.

  • ModelScan Open source

    Scans model files for code that would run when the model is loaded.

Read next

  • AI Agent: A language model given tools and allowed to decide, step by step, which ones to use to reach a goal.
  • AI Supply Chain Compromise: Tampering with the models, datasets, libraries or tools an AI system is built from.
  • Indirect Prompt Injection: Instructions hidden in content a model reads on someone else's behalf, such as a web page, an email or a document.
  • Supply Chain Compromise: Tampering with software or hardware before it reaches the victim.

Sources