Intelligent Prompt Construction for Large Language Models in Knowledge-based Visual Question Answering

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities in knowledge-based Visual Question Answering (VQA). However, existing prompt construction methods are often rigid and fail to fully exploit the reasoning potential of LLMs. To address this limitation, we propose the Intelligent Prompt Construction Framework (IPCF), which equips an autonomous agent with the ability to dynamically generate task-specific prompts. IPCF consists of a planner and a toolbox: the planner, powered

Categories

@misc{hu2026intelligent,
  title = {{Intelligent Prompt Construction for Large Language Models in Knowledge-based Visual Question Answering}},
  author = {Zhongjian Hu and Peng Yang and Dongmei Yang and Fengyuan Liu},
  year = {2026},
  month = sep,
  doi = {10.1145/3833389},
  url = {https://doi.org/10.1145/3833389}
}