Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications

Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, Danqi Chen

Abstract

Demonstrates that safety alignment in LLMs is brittle and can be undermined through simple weight pruning or low-rank modifications without any fine-tuning data.

Categories

Tags

Framework Mappings

Cite This Resource

@article{llmsec202400035,
  title = {Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications},
  author = {Yangsibo Huang and Samyak Gupta and Mengzhou Xia and Kai Li and Danqi Chen},
  year = {2024},
  journal = {ICLR 2024},
  url = {https://arxiv.org/abs/2402.05162},
}