Membership Inference
9 resourcesAttacks & Threats
Determining whether specific data was used in training
Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models
Zeyuan Chen, Yihan Ma, Xinyue Shen + 2 more
Large language models (LLMs) show strong performance across many applications, but their ability to memorize and potentially reveal training data raises serious privacy concerns. We introduce the PopQuiz Attack, a black-box membership inference attack that tests whether a model can recall specific training examples. The core idea is to turn target data into quiz-style multiple-choice questions and infer membership from the model's answers. Across six widely used LLMs (GPT-3.5, GPT-4o, LLaMA2-7b,
Membership Inference Attacks Against Video Large Language Models
Wei Song, Yuxin Cao, Ziqi Ding + 3 more
Video large language models (VideoLLMs) are increasingly trained or instruction-tuned on large-scale video--text corpora collected from heterogeneous sources, raising an immediate privacy question: can an external auditor determine whether a particular video was used during training? While membership inference attacks (MIAs) have been studied extensively for classifiers and, more recently, for text and image generation models, the VideoLLM setting remains unexplored. This setting is challenging
Towards Secure Retrieval-Augmented Generation: A Comprehensive Review of Threats, Defenses and Benchmarks
Yanming Mu, Hao Hu, Feiyang Li + 7 more
Retrieval-Augmented Generation (RAG) significantly mitigates the hallucinations and domain knowledge deficiency in large language models by incorporating external knowledge bases. However, the multi-module architecture of RAG introduces complex system-level security vulnerabilities. Guided by the RAG workflow, this paper analyzes the underlying vulnerability mechanisms and systematically categorizes core threat vectors such as data poisoning, adversarial attacks, and membership inference attacks
Prompt Stealing Attacks Against Text-to-Image Generation Models
Xinyue Shen, Yiting Qu, Michael Backes + 1 more — USENIX Security 2024
Demonstrates attacks that steal the prompts used to generate images from text-to-image models, raising IP and privacy concerns.
Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks
Vaidehi Patil, Peter Hase, Mohit Bansal — ICLR 2024
Evaluates methods for deleting sensitive information from trained LLMs, finding current unlearning approaches insufficient against determined adversaries.
Pandora's White-Box: Precise Training Data Detection and Extraction in Large Language Models
Jeffrey Cheng, Ruoxi Jia — arXiv preprint
Develops precise methods for detecting and extracting training data from LLMs when white-box access is available, with implications for copyright and privacy.
Scalable Extraction of Training Data from (Production) Language Models
Milad Nasr, Nicholas Carlini, Jonathan Hayase + 7 more — arXiv preprint
Develops a scalable attack to extract over a gigabyte of training data from semi-open and closed models including ChatGPT, at a cost of roughly $200.
Multi-step Jailbreaking Privacy Attacks on ChatGPT
Haoran Li, Dadi Guo, Wei Fan + 4 more — EMNLP 2023 Findings
Demonstrates multi-step jailbreaking attacks to extract personal information from ChatGPT, showing how sequential prompting can bypass safety measures.
Extracting Training Data from Large Language Models
Nicholas Carlini, Florian Tramer, Eric Wallace + 9 more — USENIX Security 2021
Demonstrates that large language models memorize and can be prompted to emit verbatim training data, including PII, revealing significant privacy risks.