HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, Dan Hendrycks

Abstract

Introduces HarmBench, a standardized framework for evaluating automated red teaming methods and robust refusal in LLMs with a comprehensive behavior taxonomy.

Categories

Tags

Framework Mappings

Cite This Resource

@article{llmsec202500004,
  title = {HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal},
  author = {Mantas Mazeika and Long Phan and Xuwang Yin and Andy Zou and Zifan Wang and Norman Mu and Elham Sakhaee and Nathaniel Li and Steven Basart and Bo Li and David Forsyth and Dan Hendrycks},
  year = {2024},
  journal = {ICML 2024},
  url = {https://arxiv.org/abs/2402.04249},
}