paperAugust 2026Unreviewed

The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark

Jeremy Spence, Nicholas Assaderaghi, Jinhao Zhu, Nikil Ravi, Raluca Ada Popa, Guannan Wei, Yangruibo Ding, Zhuo Zhang

Abstract

AI agents are rapidly improving in cybersecurity capabilities when the source code is available for analysis, yet much of the software most consequential to cybersecurity, including malware, firmware, and proprietary applications, is available only as binaries. Analyzing such software requires reverse engineering(RE): recovering program semantics before the analysis can be meaningfully performed. However, evaluating agentic RE poses a fundamental challenge: benchmark instances must be unseen as

Cite

@misc{spence2026next,
  title = {{The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark}},
  author = {Jeremy Spence and Nicholas Assaderaghi and Jinhao Zhu and Nikil Ravi and Raluca Ada Popa and Guannan Wei and Yangruibo Ding and Zhuo Zhang},
  year = {2026},
  month = aug,
  eprint = {2608.11469},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.11469}
}