SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation

Abstract

Prompt injection is listed as the \#1 threat to AI agents. When an agent accesses external data from websites, files, or emails, an attacker may inject a prompt into the data, saying, "Ignore all prior instructions and perform <an attacker's task>." To prevent arbitrary manipulation of agents, defenders try to train secure LLMs, which, however, still suffer from near 100% attack success rates (ASRs) against adaptive prompt injections. We note that this is because existing defensive finetuning re

Categories

Framework mappings

Suggested from the entry's categories.

@misc{peng2026secopd,
  title = {{SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation}},
  author = {Yibo Peng and Long Lian and David Wagner and Sizhe Chen},
  year = {2026},
  month = aug,
  eprint = {2608.21500},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.21500}
}