No PUN Intended: Plausible Unknown Names for Person-Centred LLM Evaluation

Abstract

Person names are widely used as prompt variables in LLM evaluations of factuality, privacy leakage, bias and abstention, but when a name's evidential status is uncontrolled, measurements may conflate memorisation, retrieval, name priors and wrong-person attribution. We operationalise an unknown name as one with plausible First-Last form, no indexed full-name evidence, and no ambiguity signals under a documented validation run, and introduce PUN (Plausible Unknown Names), a protocol for construct

Categories

Framework mappings

Suggested from the entry's categories.

@misc{staufer2026no,
  title = {{No PUN Intended: Plausible Unknown Names for Person-Centred LLM Evaluation}},
  author = {Dimitri Staufer and David Hartmann and Ibrahim Baroud},
  year = {2026},
  month = aug,
  eprint = {2608.21206},
  archivePrefix = {arXiv},
  url = {https://www.semanticscholar.org/paper/7a824f284f64b49e74d0eeccea6bf406889eab2f}
}