February 2024ReviewedOpen access
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, Dan Hendrycks
ICML 2024
Abstract
Introduces HarmBench, a standardized framework for evaluating automated red teaming methods and robust refusal in LLMs with a comprehensive behavior taxonomy.
Categories
#benchmark#red-teaming#evaluation-framework#dataset
Framework mappings
OWASP Top 10 for LLM Applications
- LLM01Prompt Injection
NIST AI Risk Management Framework
- MEASUREMeasure
Cite
@misc{mazeika2024harmbench,
title = {{HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal}},
author = {Mantas Mazeika and Long Phan and Xuwang Yin and Andy Zou and Zifan Wang and Norman Mu and Elham Sakhaee and Nathaniel Li and Steven Basart and Bo Li and David Forsyth and Dan Hendrycks},
year = {2024},
month = feb,
eprint = {2402.04249},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2402.04249}
}