Beyond Attack Success Rate: Temporal Logit Observability for LLM Safety Failures

Abstract

Attack Success Rate (ASR) evaluates each jailbreak with a single yes/no label at the end of generation, telling us whether a failure happened but not how it unfolded. Two attacks that produce equally harmful outputs may have followed completely different paths, and ASR cannot tell them apart. We make those hidden paths observable from logits alone. Temporal Logit Observability (TLO) is a training-free diagnostic that watches a compliance-refusal margin during decoding and places each model-attac

Categories

Framework mappings

Suggested from the entry's categories.

@misc{park2026beyond,
  title = {{Beyond Attack Success Rate: Temporal Logit Observability for LLM Safety Failures}},
  author = {Junyoung Park and Sunghwan Park and Seongyong Ju and Jaewoo Lee},
  year = {2026},
  month = may,
  eprint = {2605.29629},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.29629}
}