Unsafe Model Deserialization
Also called Pickle model attack.
A model file format that can run code when it is loaded, turning a download into an execution.
What it is
Some model formats store objects with Python's pickle, which can carry instructions that execute during loading. Opening such a file is equivalent to running a script from whoever made it. Weight-only formats that store tensors without code avoid this class of problem.
Why it matters
It is insecure deserialization in a new place. Data scientists' workstations and training servers often hold credentials and broad network access, which makes them valuable footholds.
In practice
Loading a shared model on a GPU server spawns a reverse shell. The model still loads and works, so nobody notices until the outbound connection is flagged.
What to look for
- Model files in pickle-based formats from outside sources.
- Child processes or network connections started by a model-loading process.
- Scanner findings for imports such as os or subprocess inside a model file.
Common mistake
Thinking a file is safe because it is "just weights". Check the format before you load it, and load untrusted models in an isolated environment.
In the field
Well-known tools where this shows up on the job. They are listed to show the landscape, not as a recommendation.
-
garak Open source
Probes language models for prompt injection, jailbreaks and data leakage.
-
PyRIT Open source
Microsoft's framework for red teaming generative AI systems.
-
ModelScan Open source
Scans model files for code that would run when the model is loaded.
Read next
- Insecure Deserialization: Rebuilding objects from untrusted data, enabling code execution.
- AI Supply Chain Compromise: Tampering with the models, datasets, libraries or tools an AI system is built from.
- Model Backdoor: A hidden behavior planted in a model that activates only when a specific trigger appears.