The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions

Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, Alex Beutel

Abstract

Proposes an instruction hierarchy for training LLMs to prioritize system prompts over user prompts over third-party content, as a defense against prompt injection.

Categories

Tags

Framework Mappings

Cite This Resource

@article{llmsec202400005,
  title = {The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions},
  author = {Eric Wallace and Kai Xiao and Reimar Leike and Lilian Weng and Johannes Heidecke and Alex Beutel},
  year = {2024},
  journal = {arXiv preprint},
  url = {https://arxiv.org/abs/2404.13208},
}