A jailbreak is an attempt to get a large language model (LLM) to ignore its safety rules and produce output it’s designed to refuse, such as harmful instructions, prohibited content or behaviour outside its intended purpose.
For a company running a chatbot or AI feature, jailbreaks matter less because of what the model knows and more because of what a jailbroken model can say or do under your brand.
Jailbreak vs prompt injection
The two overlap and the terms are often mixed up:
- Jailbreaking targets the model’s safety and behaviour rules: getting it to produce what it’s trained or instructed to refuse.
- Prompt injection targets the application’s instructions: getting the system to follow an attacker’s instructions instead of the developer’s, often to leak data or take actions.
A jailbreak is often done through prompt injection, and in the OWASP Top 10 for LLM Applications jailbreaking is treated as a form of prompt injection.
Common jailbreak techniques (at a high level)
Security teams should recognise the patterns. The specifics change constantly:
- Role-play and personas: asking the model to act as a character “without restrictions”.
- Hypotheticals and fiction: wrapping a request in a story, a game or “for research purposes”.
- Gradual escalation: many harmless steps that slowly move the conversation over the line.
- Obfuscation: encoding requests, splitting them up, using other languages or unusual formatting to slip past filters.
- Instruction override: claiming special authority, such as “I’m the developer” or “debug mode”.
Model providers continually train against known techniques, and attackers continually find new ones. Assume a determined user can eventually jailbreak any current model.
Why it matters for businesses
- Reputation: screenshots of your branded chatbot producing offensive or dangerous content spread fast.
- Legal exposure: a chatbot making false promises, discriminatory statements or giving regulated advice, such as legal, medical or financial, can create liability.
- Policy bypass: a jailbroken support bot may offer refunds, discounts or exceptions it was told never to give.
- Escalation: combined with tools or data access, a jailbreak can become data leakage or unauthorised actions. See AI agent security.
How to limit jailbreak risk in your AI apps
- Don’t rely on the model alone. Enforce business rules, such as refunds, pricing and data access, in code and systems outside the model.
- Narrow the scope. A bot that only handles order tracking is much harder to misuse than a general assistant with your logo on it.
- Add input and output filtering with moderation or guardrail tools to catch harmful requests and responses.
- Limit privileges so a jailbroken bot can’t access data or take actions beyond its job.
- Write clear disclaimers so users know the bot can be wrong and what it can’t commit to on your behalf.
- Red-team before launch, including jailbreak attempts relevant to your brand and sector.
- Monitor conversations (lawfully) for abuse patterns, and have a way to shut the bot down fast.
The bottom line
You can’t make a language model unjailbreakable today. You can make sure a jailbreak doesn’t give anyone anything valuable: no data, no actions, no promises that bind your company.
Official sources
- OWASP Top 10 for LLM Applications: the OWASP GenAI Security Project’s list of the main LLM risks
- MITRE ATLAS: a knowledge base of real attack techniques against AI systems
- NIST AI 100-2 E2025, Adversarial Machine Learning: NIST’s taxonomy of attacks on AI systems and their mitigations
More primary sources, with a short note on each, are on our Sources page.
Last reviewed October 2026. This article is general information for defensive purposes.