Inferring Hidden User Models from the Behavior of Personalized LLM Agents

Abstract

Recent personalized LLM agents increasingly transform information retained in memory into compressed or structured representations, which we call user models, to guide later decisions. When source wording is removed from the state reachable through the ordinary interface, these models are commonly treated as more privacy-preserving because direct memory-extraction attacks lose the text they target. Yet we argue that user models expose a new attack surface because an attacker can still recover th

Categories

@misc{li2026inferring,
  title = {{Inferring Hidden User Models from the Behavior of Personalized LLM Agents}},
  author = {Haoyang Li and Yaxin Xiao and Qingqing Ye and Huadi Zheng and Haibo Hu},
  year = {2026},
  month = sep,
  eprint = {2609.03815},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.03815}
}