Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications

Abstract

Demonstrates that safety alignment in LLMs is brittle and can be undermined through simple weight pruning or low-rank modifications without any fine-tuning data.

Categories

Framework mappings

@inproceedings{wei2024assessing,
  title = {{Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications}},
  author = {Boyi Wei and Kaixuan Huang and Yangsibo Huang and Tinghao Xie and Xiangyu Qi and Mengzhou Xia and Prateek Mittal and Mengdi Wang and Peter Henderson},
  year = {2024},
  booktitle = {ICML 2024},
  eprint = {2402.05162},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.05162}
}