Once Poisoned, Arbitrarily Controlled: A Programmable Backdoor in VLMs

Abstract

Existing vision-language model (VLM) backdoors are usually treated as static vulnerabilities: one-to-one and N-to-N attacks bind one or more triggers to a finite set of targets before victim training. This assumption substantially underestimates the threat. We show that a single poisoning phase can implant a programmable backdoor into a VLM, allowing an attacker to choose previously unseen target-caption semantics at inference time and synthesize corresponding stealthy triggers on demand. Unlike

Categories

Framework mappings

Suggested from the entry's categories.

@misc{lin2026once,
  title = {{Once Poisoned, Arbitrarily Controlled: A Programmable Backdoor in VLMs}},
  author = {Tao Lin and Gaojie Jin and Zongxi Liu and Peng Wu and Lijia Yu},
  year = {2026},
  month = aug,
  eprint = {2608.10959},
  archivePrefix = {arXiv},
  url = {https://www.semanticscholar.org/paper/d72f36000b19aeeb672f3fc41a15e33915bc8ca3}
}