Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models

Abstract

Adversarial robustness evaluations of large language models (LLMs) typically report attack success rate (ASR) under fixed query budgets, implicitly treating all attacks as equally costly. In practice, the computational expense of different attack strategies can vary by orders of magnitude. Consequently, ASR at a fixed budget can obscure the true effort required to jailbreak a model, thereby making it hard to determine whether an attack's cost justifies its payoff to the attacker. We propose a co

Categories

Framework mappings

Suggested from the entry's categories.

@misc{ehghaghi2026risk,
  title = {{Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models}},
  author = {Malikeh Ehghaghi and Boglárka Ecsedi and Marsha Chechik and Colin Raffel},
  year = {2026},
  month = jun,
  eprint = {2606.11409},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.11409}
}