paperOctober 2025Unreviewed

SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents via Distilled Structured Reasoning

Kai Zhou, Ahmed Elgohary, S. M. Iftekhar, ♣. Amin, Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, Yun-ing Mao, Mart, Daya Guo, Dejian Yang, Haowei Zhang, Jun-Mei Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Weiyang Guo, Ze-Yun Shi, Zhuo Li, Yequan Wang, Xiaogeng Liu, Peiran Li, Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, P. McDaniel, Huan Sun, Bo Li, Yanjiang Liu, Shuheng Zhou, Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Ya-Ting Lu, Yankai Lin, X. Cong, Xiangru Tang, Mikayel Samvelyan, S. Raparthy, Andrei Lupu, Eric Hambro, A. Markosyan, Manish Bhatt, Chejian Xu, Mintong Kang, Jiawei Zhang, Zeyi Liao, Lingbo Mo, Mengqi Yuan, An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bo Yu, Chang Gao, Shunyu Yao, H. Chen, John Yang, Karthik R. Narasimhan, Webshop, Andy Zhou, Kevin Wu, Francesco Pinto, Zhaorun Chen, Yi Zeng, Yu Yang, Shuang Yang, Sanmi Koyejo, James Zou, Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr

Conference of the European Chapter of the Association for Computational Linguistics

View resource

Abstract

The ability of LLM agents to plan and invoke tools exposes them to new safety risks, making a comprehensive red-teaming system crucial for discovering vulnerabilities and ensuring their safe deployment. We present SIRAJ: a generic red-teaming framework for arbitrary black-box LLM agents. We employ a dynamic two-step process that starts with an agent definition and generates diverse seed test cases that cover various risk outcomes, tool-use trajectories, and risk sources. Then, it iteratively con

Framework mappings

NIST AI Risk Management Framework

MEASUREMeasure

Suggested from the entry's categories.

Cite

@inproceedings{zhou2025siraj,
  title = {{SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents via Distilled Structured Reasoning}},
  author = {Kai Zhou and Ahmed Elgohary and S. M. Iftekhar and ♣. Amin and Suyu Ge and Chunting Zhou and Rui Hou and Madian Khabsa and Yi-Chia Wang and Qifan Wang and Jiawei Han and Yun-ing Mao and Mart and Daya Guo and Dejian Yang and Haowei Zhang and Jun-Mei Song and Ruoyu Zhang and Runxin Xu and Qihao Zhu and Shirong Ma and Weiyang Guo and Ze-Yun Shi and Zhuo Li and Yequan Wang and Xiaogeng Liu and Peiran Li and Edward Suh and Yevgeniy Vorobeychik and Zhuoqing Mao and Somesh Jha and P. McDaniel and Huan Sun and Bo Li and Yanjiang Liu and Shuheng Zhou and Yujia Qin and Shihao Liang and Yining Ye and Kunlun Zhu and Lan Yan and Ya-Ting Lu and Yankai Lin and X. Cong and Xiangru Tang and Mikayel Samvelyan and S. Raparthy and Andrei Lupu and Eric Hambro and A. Markosyan and Manish Bhatt and Chejian Xu and Mintong Kang and Jiawei Zhang and Zeyi Liao and Lingbo Mo and Mengqi Yuan and An Yang and Anfeng Li and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bo Yu and Chang Gao and Shunyu Yao and H. Chen and John Yang and Karthik R. Narasimhan and Webshop and Andy Zhou and Kevin Wu and Francesco Pinto and Zhaorun Chen and Yi Zeng and Yu Yang and Shuang Yang and Sanmi Koyejo and James Zou and Andy Zou and Zifan Wang and Nicholas Carlini and Milad Nasr},
  year = {2025},
  month = oct,
  booktitle = {Conference of the European Chapter of the Association for Computational Linguistics},
  eprint = {2510.26037},
  archivePrefix = {arXiv},
  doi = {10.48550/arXiv.2510.26037},
  url = {https://www.semanticscholar.org/paper/910c759401877c93f37b5114d069108ea7c140e7}
}