Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Manish Bhatt, Sahana Chennabasappa, Cyrus Nikolaidis, Shengye Wan, Ivan Evtimov

Abstract

Introduces CyberSecEval, a benchmark for evaluating the cybersecurity risks of LLM code generation, including insecure code suggestions.

Categories

Tags

Framework Mappings

Cite This Resource

@article{llmsec202400052,
  title = {Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models},
  author = {Manish Bhatt and Sahana Chennabasappa and Cyrus Nikolaidis and Shengye Wan and Ivan Evtimov},
  year = {2024},
  journal = {arXiv preprint},
  url = {https://arxiv.org/abs/2312.04724},
}