Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory

Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, Yejin Choi

Abstract

Evaluates LLM privacy behavior through the lens of contextual integrity theory, finding significant mismatches between LLM norms and human privacy expectations.

Categories

Tags

Framework Mappings

Cite This Resource

@article{llmsec202400015,
  title = {Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory},
  author = {Niloofar Mireshghallah and Hyunwoo Kim and Xuhui Zhou and Yulia Tsvetkov and Maarten Sap and Reza Shokri and Yejin Choi},
  year = {2024},
  journal = {ICLR 2024},
  url = {https://arxiv.org/abs/2310.17884},
}