Not What You Asked For: Typographic Attacks in Household Robot Manipulation

Abstract

Open-vocabulary embodied AI agents increasingly rely on vision-language models such as CLIP for object perception and task grounding. However, the shared embedding space that enables this flexibility introduces a structural vulnerability to typographic attacks, where printed text in a physical scene semantically overrides visual judgment. While prior work has quantified this threat in static 2D benchmarks and 3D navigation tasks, its impact on the full Sense-Plan-Act pipeline of household robot

Categories

@misc{iranmanesh2026not,
  title = {{Not What You Asked For: Typographic Attacks in Household Robot Manipulation}},
  author = {Ali Iranmanesh and Peng Liu},
  year = {2026},
  month = may,
  eprint = {2605.18593},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.18593}
}