CyberKit

Jailbreak

Also called LLM jailbreak, AML.T0054.

A prompt that gets a model to produce output its safety training is meant to refuse.

What it is

An attempt to bypass a model's built-in safety behavior, usually through role-play, hypotheticals, encoding or long chains of setup. Where prompt injection hijacks an application's instructions, a jailbreak targets the model's own restrictions. MITRE ATLAS tracks it as AML.T0054, LLM Jailbreak.

Why it matters

A jailbroken model in a business application can produce content that creates legal or safety exposure in the company's name. It is also often the first step before a more targeted injection.

In practice

A user asks a customer-facing assistant to pretend it is a character with no rules, then asks the character for instructions the assistant would normally refuse.

What to look for

  • Prompts built around role-play, fiction or a claimed test mode.
  • Requests encoded in Base64, another language or split across turns.
  • A refusal followed by many rephrasings of the same request.

Common mistake

Treating a model's refusal as a security control. Refusals are probabilistic; anything that must never happen needs a control outside the model.

In the field

Well-known tools where this shows up on the job. They are listed to show the landscape, not as a recommendation.

  • garak Open source

    Probes language models for prompt injection, jailbreaks and data leakage.

  • PyRIT Open source

    Microsoft's framework for red teaming generative AI systems.

  • ModelScan Open source

    Scans model files for code that would run when the model is loaded.

Read next

  • Prompt Injection: Text crafted to make a language model ignore its instructions and follow the attacker's instead.
  • Guardrails: Checks around a model that filter what goes in, what comes out, and what it is allowed to do.
  • AI Red Teaming: Attacking an AI system on purpose, before someone else does, to find how it fails.

Sources