Confidently Wrong: Severity-Aware Calibration of Prompt-Injection Detectors under Attack Shift

Abstract

Prompt-injection detectors are deployed as guards: a model scores an input and a downstream system trusts or blocks it on that score. I study the confidence of these scores, not only their accuracy, when the attack distribution shifts away from the clean benchmark on which the operating point was chosen. I evaluate three released detectors, ProtectAI-v2 and two Prompt-Guard-2 checkpoints, at a single source-calibrated threshold that I freeze and transport across five shifts. I report a severity

Categories

Framework mappings

Suggested from the entry's categories.

@misc{biswas2026confidently,
  title = {{Confidently Wrong: Severity-Aware Calibration of Prompt-Injection Detectors under Attack Shift}},
  author = {Md Anas Biswas},
  year = {2026},
  month = jun,
  eprint = {2606.22659},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.22659}
}