Skip to content
Search
report2025ReviewedOpen access

Lessons From Red Teaming 100 Generative AI Products

Blake Bullwinkel, Amanda Minnich, Shiven Chawla, Gary Lopez, Martin Pouliot, Whitney Maxwell, Joris de Gruyter, Katherine Pratt, Saphir Qi, Nina Chikanov, Roman Lutz, Raja Sekhar Rao Dheekonda, Bolor-Erdene Jagdagdorj, Eugenia Kim, Justin Song, Keegan Hines, Daniel Jones, Giorgio Severi, Richard Lundeen, Sam Vaughan, Victoria Westerhoff, Pete Bryan, Ram Shankar Siva Kumar, Yonatan Zunger, Chang Kawaguchi, Mark Russinovich

arXiv preprint

View resourcePDF

Abstract

Shares lessons from Microsoft's AI red team operations including methodology, tooling, common failure modes, and best practices.

Categories

Red TeamingIndustry Reports
#Microsoft#red-team#methodology#lessons-learned

Framework mappings

NIST AI Risk Management Framework
  • MEASUREMeasure
  • MANAGEManage

Cite

@techreport{bullwinkel2025lessons,
  title = {{Lessons From Red Teaming 100 Generative AI Products}},
  author = {Blake Bullwinkel and Amanda Minnich and Shiven Chawla and Gary Lopez and Martin Pouliot and Whitney Maxwell and Joris de Gruyter and Katherine Pratt and Saphir Qi and Nina Chikanov and Roman Lutz and Raja Sekhar Rao Dheekonda and Bolor-Erdene Jagdagdorj and Eugenia Kim and Justin Song and Keegan Hines and Daniel Jones and Giorgio Severi and Richard Lundeen and Sam Vaughan and Victoria Westerhoff and Pete Bryan and Ram Shankar Siva Kumar and Yonatan Zunger and Chang Kawaguchi and Mark Russinovich},
  year = {2025},
  institution = {arXiv preprint},
  eprint = {2501.07238},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.07238}
}