AI Supply Chain Compromise
Also called AML.T0010.
Tampering with the models, datasets, libraries or tools an AI system is built from.
What it is
An attack on any upstream part of an AI system: a pre-trained model from a public hub, a dataset, a training library, or a plugin an agent installs. MITRE ATLAS tracks it as AML.T0010, AI Supply Chain Compromise, with sub-techniques for each kind of artifact.
Why it matters
Teams that would never run an unsigned binary routinely download model weights and datasets from strangers. The same supply chain discipline applies: provenance, hashes, scanning and least privilege.
In practice
A developer pulls a popular-looking model from a public hub. The file is a near-copy of a real one, uploaded under a similar name, and it runs code on load.
What to look for
- Models or datasets pulled by name with no pinned hash or version.
- Publisher names that imitate well-known organizations.
- Build or training jobs making unexpected outbound connections.
Common mistake
Scanning the code that loads a model but not the model file. A model file can be the payload.
In the field
Well-known tools where this shows up on the job. They are listed to show the landscape, not as a recommendation.
-
garak Open source
Probes language models for prompt injection, jailbreaks and data leakage.
-
PyRIT Open source
Microsoft's framework for red teaming generative AI systems.
-
ModelScan Open source
Scans model files for code that would run when the model is loaded.
Read next
- Unsafe Model Deserialization: A model file format that can run code when it is loaded, turning a download into an execution.
- Model Backdoor: A hidden behavior planted in a model that activates only when a specific trigger appears.
- Supply Chain Compromise: Tampering with software or hardware before it reaches the victim.
- Typosquatting: Registering a domain that's a common misspelling of a legitimate one.
Sources
- NSA and partners: AI/ML Supply Chain Risks and Mitigations
- MITRE ATLAS: AML.T0010, AI Supply Chain Compromise
- NSA, CISA and partners: AI Data Security
- NIST AI 100-2 E2025: Adversarial Machine Learning, a Taxonomy and Terminology
- NIST AI Risk Management Framework
- CISA, NSA and partners: Deploying AI Systems Securely