CyberKit

Unsafe Model Deserialization

Also called Pickle model attack.

A model file format that can run code when it is loaded, turning a download into an execution.

What it is

Some model formats store objects with Python's pickle, which can carry instructions that execute during loading. Opening such a file is equivalent to running a script from whoever made it. Weight-only formats that store tensors without code avoid this class of problem.

Why it matters

It is insecure deserialization in a new place. Data scientists' workstations and training servers often hold credentials and broad network access, which makes them valuable footholds.

In practice

Loading a shared model on a GPU server spawns a reverse shell. The model still loads and works, so nobody notices until the outbound connection is flagged.

What to look for

  • Model files in pickle-based formats from outside sources.
  • Child processes or network connections started by a model-loading process.
  • Scanner findings for imports such as os or subprocess inside a model file.

Common mistake

Thinking a file is safe because it is "just weights". Check the format before you load it, and load untrusted models in an isolated environment.

In the field

Well-known tools where this shows up on the job. They are listed to show the landscape, not as a recommendation.

  • garak Open source

    Probes language models for prompt injection, jailbreaks and data leakage.

  • PyRIT Open source

    Microsoft's framework for red teaming generative AI systems.

  • ModelScan Open source

    Scans model files for code that would run when the model is loaded.

Read next

Sources