Cyb-LLM: A Unified Benchmark for Evaluating LLMs in Cyber Attack & Defense

Abstract

There is a growing use of large language models LLMs in security-related workflows. There is a gap in existing benchmarks for assessing dual-use capabilities of LLMs under safety constraints. We introduce a new, unified benchmark called CybLLM, which encompasses both offensive tasks (e.g., phishing generation, exploit request) and defensive tasks (e.g., secure coding, malware, and triage with built-in safety alignment metrics. Our framework integrates offense, Defense, and utility assessments us

Categories

@inproceedings{segireddy2026cybllm,
  title = {{Cyb-LLM: A Unified Benchmark for Evaluating LLMs in Cyber Attack \& Defense}},
  author = {Akshitha Segireddy},
  year = {2026},
  month = feb,
  booktitle = {2026 International Conference on Visual Analytics and Data Visualization (ICVADV)},
  doi = {10.1109/ICVADV67766.2026.11470531},
  url = {https://www.semanticscholar.org/paper/a8cef9a306f824b558108d45011f675c2244940b}
}