Skip to content
Search
paperNovember 2023ReviewedOpen access

Scalable Extraction of Training Data from (Production) Language Models

Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo, Eric Wallace, Florian Tramer, Katherine Lee

arXiv preprint

View resourcePDF

Abstract

Develops a scalable attack to extract over a gigabyte of training data from semi-open and closed models including ChatGPT, at a cost of roughly $200.

Categories

Membership InferenceModel Extraction
#training-data-extraction#ChatGPT#production-model

Framework mappings

OWASP Top 10 for LLM Applications
  • LLM02Sensitive Information Disclosure
MITRE ATLAS
  • AML.T0024Exfiltration via AI Inference API
  • AML.T0057LLM Data Leakage

Cite

@misc{nasr2023scalable,
  title = {{Scalable Extraction of Training Data from (Production) Language Models}},
  author = {Milad Nasr and Nicholas Carlini and Jonathan Hayase and Matthew Jagielski and A. Feder Cooper and Daphne Ippolito and Christopher A. Choquette-Choo and Eric Wallace and Florian Tramer and Katherine Lee},
  year = {2023},
  month = nov,
  eprint = {2311.17035},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2311.17035}
}