The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions

Abstract

Proposes an instruction hierarchy for training LLMs to prioritize system prompts over user prompts over third-party content, as a defense against prompt injection.

Categories

Framework mappings

@misc{wallace2024instruction,
  title = {{The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions}},
  author = {Eric Wallace and Kai Xiao and Reimar Leike and Lilian Weng and Johannes Heidecke and Alex Beutel},
  year = {2024},
  month = apr,
  eprint = {2404.13208},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.13208}
}