Guardrail bypass and alignment subversion techniques
Newest first · 4 reviewed on this page
Search insteadShi-Liang Xiao
Gradient-based jailbreak suffix optimization methods typically update the suffix by retaining the candidate with the lowest current loss. We show that this seemingly natural design is fundamentally myopic: candidates that look better under the current-step proxy often fail to…
Yu Zhe, Yiting Tan, Jun-Hao Wei +1
Model merging enables combining multiple fine-tuned models without additional training, but its safety implications remain poorly understood. Prior work primarily attributes merging risks to unsafe constituent models, implicitly assuming that merging individually aligned models…
Afshin Orojlooyjadid, Hitesh Laxmichand Patel
Large Language Models (LLMs) are increasingly deployed in real-world applications, yet they remain vulnerable to generating harmful content. From adversarial jailbreaks that bypass safety filters to implicit hate that evades detection, the range of risks these models pose…
Qiuxiang Li, Ke Xu, Yubin Qu +1
Yang Liu, Bin Chong, Wenkai Yang +9
Large language models (LLMs) are vulnerable to multi-turn jailbreak attacks that progressively manipulate conversation context. Existing certified robustness methods are limited to single-turn inputs; naive multi-turn composition yields bounds that degrade exponentially in the…
Wenzheng Jiang, Xuan-Kun Rong, Yuan-Zhao Zhai +2
While multimodal large language models (MLLMs) extend model capabilities beyond text, they also make safety alignment increasingly challenging. Multimodal safety alignment methods must address cross-modal jailbreaks, safety-awareness failures, and over-sensitive refusals.…
Ling Zhou, Yihao Huang, Jinglin Sun +4
Large vision-language models (LVLMs) have achieved remarkable progress in video understanding and reasoning. Despite extensive studies on text- and image-based jailbreaks, video jailbreaks against LVLMs remain largely unexplored. Existing video jailbreak methods mainly…
Michael Tchuindjang, Nathan Duran, Phil Legg +1
Existing multi-turn jailbreak evaluation methods increasingly rely on large language models (LLMs) as automated judges to reduce the cost and scalability limitations of human assessment. However, recent studies show that LLM-based evaluators can diverge from human judgments…
Sana Mourad, E. E. Abdallah, Mohammad Ababneh
Current language model deployments face growing security challenges from prompt-based attacks, including jailbreaks, direct and indirect prompt injection, and instruction hijacking, which often evade traditional rule-based safeguards. As these models are increasingly integrated…
Shun Zhang, Ying Ding, Yanxu Mao
With the rapid development and widespread deployment of large language models (LLMs), the security and robustness of these models have emerged as critical research topics. Among various threats, jailbreak attacks, which aim to circumvent built‐in safety mechanisms, have garnered…
Jiawei Liu, Jiacheng Guo, Tian Zhang +4
Foundation models are increasingly used for perception, reasoning, planning, and action generation in embodied agents, creating security risks that can propagate from digital inputs to physical behavior. Existing surveys often organize threats by mechanisms such as jailbreaks,…
Md Messal Monem Miah, Adrita Anika, Zhiyuan Yu +1
Multi-turn jailbreak attacks have emerged as a critical safety threat to LLMs, as harmful objectives are decomposed across a sequence of apparently benign turns to bypass guardrails. Existing defenses lack the reasoning capacity to identify evolving manipulation patterns, often…
Wei Zhao, Zhe Li, Peixin Zhang +1
Neuron- and path-level interventions offer the finest-grained route to defending large language models (LLMs) against jailbreak attacks, yet existing methods fall short of this promise, i.e., they often compromise model utility significantly. Specifically, one line of work…
Fang-Zhou Chen, Shiji Zhao, Mengyan Wang +4
Large language models (LLMs) remain vulnerable to harmful requests and jailbreak attacks. Parameter-efficient safety alignment methods based on prompt tuning typically rely on a single global prompt or externally selected prompt modules. Such static designs struggle to maintain…
Caoyuan Ma, Wen-Pu Liu, Weichu Xie +12
Large vision-language models (LVLMs) remain vulnerable to jailbreak attacks that exploit visual inputs to bypass safety alignment inherited from their language backbones. We propose SafeCap, a reinforcement-learning framework that aligns LVLMs through learned self-captioning.…
Chuqi Lin, S. Sondhi, Xiao-Liang Qi
Sparse autoencoders (SAEs) have helped uncover mechanistic explanations for LLM behaviours such as reasoning, jailbreaking etc., via understanding the corresponding task-relevant and causally effective features. To evaluate such mechanistic explanations, downstream studies must…
Adam Ait Hsine, A. Arabo
The deployment of large language models (LLMs) in real-world applications introduces a compounding security problem: detecting adversarial inputs such as prompt injection and jailbreak-driven data leakage while simultaneously preventing the detection mechanism itself from…
Ming Cong, Jingyin Chen, Bin Liu +4
Deployed LLM safety guardrails are predominantly static: trained once and frozen at release, while new jailbreak techniques and previously un-addressed harmful categories emerge within days, leaving the defense perpetually a step behind. We present SESG (Self-Evolving Safety…
Yu Ma, Hongli Shi, Jing Li +2
Model merging has become the default way to give an aligned language model new skills without retraining: a practitioner folds task vectors from math, code, or domain specialists into a safety-aligned base using task arithmetic, TIES, or DARE. This convenience is known to carry…
Lu Lin, Jinghui Chen, Ting Wang +4
Large language models (LLMs) are increasingly embedded as core components of data-centric systems, supporting analytical decision making, and automated reasoning over large-scale, heterogeneous datasets. Yet their deployment in open-world environments raises fundamental…
Alina Klerings, Jannik Brinkmann, Heiner Stuckenschmidt +1
Large language models typically undergo post-training to align them with safety policies but there exist many sophisticated jailbreaks that sidestep established safeguards. For instance, prior work by Andriushchenko et al. (2025) has found that changing the grammatical tense…
Hujian Zhu, Yihao Huang, Felix Juefei-Xu +5
Foundation models have achieved remarkable success across diverse tasks, but they remain vulnerable. To investigate such vulnerabilities, semantic-shift jailbreaks have recently emerged as a promising attack paradigm. They bypass explicit safety mechanisms by replacing harmful…
Patrick Chao, Alexander Robey, Edgar Dobriban +3
Uses an attacker LLM to automatically generate jailbreak prompts through iterative refinement achieving high success with only black-box access.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion
Shows that adaptive adversaries can bypass most proposed jailbreak defenses, highlighting the arms race between attacks and defenses.
Chetan Pathade
Large Language Models (LLMs) are increasingly integrated into consumer and enterprise applications. Despite their capabilities, they remain susceptible to adversarial attacks such as prompt injection and jailbreaks that override alignment safeguards. This paper provides a…
Zixuan Weng, Xiaolong Jin, Jinyuan Jia +1
Ensuring AI safety is crucial as large language models become increasingly integrated into real-world applications. A key challenge is jailbreak, where adversarial prompts bypass built-in safeguards to elicit harmful disallowed outputs. Inspired by psychological foot-in-the-door…
Weixiang Zhao, Jiahe Guo, Yulin Hu +8
Despite extensive efforts in safety alignment, large language models (LLMs) remain vulnerable to jailbreak attacks. Activation steering offers a training-free defense method but relies on fixed steering coefficients, resulting in suboptimal protection and increased false…
William Hackett, Lewis Birch, Stefan Trawicki +2
Large Language Models (LLMs) guardrail systems are designed to protect against prompt injection and jailbreak attacks. However, they remain vulnerable to evasion techniques. We demonstrate two approaches for bypassing LLM prompt injection and jailbreak detection systems via…
Yanxu Mao, Tiehan Cui, Peipei Liu +2
Large language models (LLMs) are rapidly evolving from single-modal systems to multimodal LLMs and intelligent agents, significantly expanding their capabilities while introducing increasingly severe security risks. This paper presents a systematic survey of the growing…
Piercosma Bisconti, Matteo Prandi, Federico Pierucci +7
We present evidence that adversarial poetry functions as a universal single-turn jailbreak technique for Large Language Models (LLMs). Across 25 frontier proprietary and open-weight models, curated poetic prompts yielded high attack-success rates (ASR), with some providers…
Shaopeng Fu, Liang Ding, Di Wang
Jailbreak attacks against large language models (LLMs) aim to induce harmful behaviors in LLMs through carefully crafted adversarial prompts. To mitigate attacks, one way is to perform adversarial training (AT)-based alignment, i.e., training LLMs on some of the most adversarial…
Daniel Schwartz, Dmitriy Bespalov, Zhe Wang +2
As large language models (LLMs) become increasingly prevalent, ensuring their robustness against adversarial misuse is crucial. This paper introduces the GAP (Graph of Attacks with Pruning) framework, an advanced approach for generating stealthy jailbreak prompts to evaluate and…
Yucheng Li, Surin Ahn, Huiqiang Jiang +3
Large language models (LLMs) have achieved widespread adoption across numerous applications. However, many LLMs are vulnerable to malicious attacks even after safety alignment. These attacks typically bypass LLMs' safety guardrails by wrapping the original malicious instructions…
Xiaohu Li, Yunfeng Ning, Zepeng Bao +3
Security alignment enables the Large Language Model (LLM) to gain the protection against malicious queries, but various jailbreak attack methods reveal the vulnerability of this security mechanism. Previous studies have isolated LLM jailbreak attacks and defenses. We analyze the…
Weiliang Zhao, Jinjun Peng, Daniel Ben-Levi +2
The proliferation of powerful large language models (LLMs) has necessitated robust safety alignment, yet these models remain vulnerable to evolving adversarial attacks, including multi-turn jailbreaks that iteratively search for successful queries. Current defenses, which are…
Xin Wei Chia, Jonathan Pan
Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks, yet they remain vulnerable to adversarial manipulations such as jailbreaking via prompt injection attacks. These attacks bypass safety mechanisms to generate restricted or harmful…
Hanbin Hong, Shuangqiao Wu, Shuya Feng +6
Large Language Models (LLMs) are increasingly used as interfaces to information, code, and real-world services, making prompt-level security failures a practical concern. Although jailbreak attacks, defenses, datasets, and automated judgers have advanced rapidly, evaluation…
Mohamed Ahmed, Mohamed Abdelmouty, Mingyu Kim +3
The advancement of Pre-Trained Language Models (PTLMs) and Large Language Models (LLMs) has led to their widespread adoption across diverse applications. Despite their success, these models remain vulnerable to attacks that exploit their inherent weaknesses to bypass safety…
Ruixuan Huang, Xunguang Wang, Zongjie Li +2
Despite the growing interest in jailbreak methods as an effective red-teaming tool for building safe and responsible large language models (LLMs), flawed evaluation system designs have led to significant discrepancies in their effectiveness assessments. We conduct a systematic…
Pavlos Ntais
Large language models (LLMs) remain vulnerable to sophisticated prompt engineering attacks that exploit contextual framing to bypass safety mechanisms, posing significant risks in cybersecurity applications. We introduce Jailbreak Mimicry, a systematic methodology for training…
Wenpeng Xing, Mohan Li, Chunqiang Hu +3
While Large Language Models (LLMs) have achieved remarkable progress, they remain vulnerable to jailbreak attacks. Existing methods, primarily relying on discrete input optimization (e.g., GCG), often suffer from high computational costs and generate high-perplexity prompts that…
Jiaming Hu, Haoyu Wang, Debarghya Mukherjee +1
Jailbreak attacks pose a serious challenge to the safe deployment of large language models (LLMs). We introduce CCFC (Core&Core-Full-Core), a dual-track, prompt-level defense framework designed to mitigate LLMs'vulnerabilities from prompt injection and structure-aware jailbreak…
Gil Goren, Shahar Katz, Lior Wolf
Large Language Models (LLMs) are vulnerable to adversarial attacks that bypass safety guidelines and generate harmful content. Mitigating these vulnerabilities requires defense mechanisms that are both robust and computationally efficient. However, existing approaches either…
Xurui Li, Kaisong Song, Rui Zhu +2
Large Language Models (LLMs) have developed rapidly in web services, delivering unprecedented capabilities while amplifying societal risks. Existing works tend to focus on either isolated jailbreak attacks or static defenses, neglecting the dynamic interplay between evolving…
Shreya Dubey, H. Lamkuche
Large Language Models (LLMs) are increasingly used across diverse applications, raising security concerns, especially regarding jail-break attacks that bypass safety measures. This paper presents a systematic security evaluation of LLMs, exploring the effectiveness of jail-break…
Dominik Schwarz
The security of Large Language Model (LLM) applications is fundamentally challenged by "formfirst" attacks like prompt injection and jailbreaking, where malicious instructions are embedded within user inputs. Conventional defenses, which rely on post hoc output filtering, are…
Kai Hu, Abhinav Aggarwal, Mehran Khodabandeh +6
This paper introduces Jailbreak-Zero, a novel red teaming methodology that shifts the paradigm of Large Language Model (LLM) safety evaluation from a constrained example-based approach to a more expansive and effective policy-based framework. By leveraging an attack LLM to…
Alexander Wei, Nika Haghtalab, Jacob Steinhardt
Analyzes failure modes of LLM safety training, identifying two broad categories: competing objectives and mismatched generalization, demonstrating attacks that exploit each.
Weidi Luo, Siyuan Ma, Xiaogeng Liu +2
With the rapid advancements in Multimodal Large Language Models (MLLMs), securing these models against malicious inputs while aligning them with human values has emerged as a critical challenge. In this paper, we investigate an important and unexplored question of whether…
Xiaogeng Liu, Nan Xu, Muhao Chen +1
Proposes AutoDAN, a method for automatically generating stealthy jailbreak prompts that are semantically meaningful and can bypass perplexity-based defenses.