Skip to content
Search
paperJanuary 2024ReviewedOpen access

R-Judge: Benchmarking Safety Risk Awareness for LLM Agents

Tongxin Yuan, Zhiwei He, Lingzhong Dong, Yiming Wang, Ruijie Zhao, Tian Xia, Lizhen Xu, Binglin Zhou, Fangqi Li, Zhuosheng Zhang, Rui Wang, Gongshen Liu

EMNLP 2024

View resourcePDF

Abstract

Introduces R-Judge benchmark for evaluating whether LLM agents can identify safety risks in agentic scenarios involving tool use and multi-step reasoning.

Categories

Agentic ThreatsBenchmarks & EvaluationHuman-in-the-Loop
#agent-safety#benchmark#risk-awareness

Framework mappings

OWASP Top 10 for LLM Applications
  • LLM06Excessive Agency

Cite

@inproceedings{yuan2024rjudge,
  title = {{R-Judge: Benchmarking Safety Risk Awareness for LLM Agents}},
  author = {Tongxin Yuan and Zhiwei He and Lingzhong Dong and Yiming Wang and Ruijie Zhao and Tian Xia and Lizhen Xu and Binglin Zhou and Fangqi Li and Zhuosheng Zhang and Rui Wang and Gongshen Liu},
  year = {2024},
  month = jan,
  booktitle = {EMNLP 2024},
  eprint = {2401.10019},
  archivePrefix = {arXiv},
  doi = {10.18653/v1/2024.findings-emnlp.79},
  url = {https://arxiv.org/abs/2401.10019}
}