Skip to content
Search
paper2021ReviewedOpen access

Extracting Training Data from Large Language Models

Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, Colin Raffel

USENIX Security 2021

View resourcePDF

Abstract

Demonstrates that large language models memorize and can be prompted to emit verbatim training data, including PII, revealing significant privacy risks.

Categories

Membership InferenceDifferential Privacy
#memorization#training-data-extraction#privacy

Framework mappings

OWASP Top 10 for LLM Applications
  • LLM02Sensitive Information Disclosure
MITRE ATLAS
  • AML.T0024Exfiltration via AI Inference API
  • AML.T0057LLM Data Leakage

Cite

@inproceedings{carlini2021extracting,
  title = {{Extracting Training Data from Large Language Models}},
  author = {Nicholas Carlini and Florian Tramer and Eric Wallace and Matthew Jagielski and Ariel Herbert-Voss and Katherine Lee and Adam Roberts and Tom Brown and Dawn Song and Ulfar Erlingsson and Alina Oprea and Colin Raffel},
  year = {2021},
  booktitle = {USENIX Security 2021},
  eprint = {2012.07805},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2012.07805}
}