Skip to content
Search
paperJune 2026Unreviewed

Evaluating LLMs for Real-World Web Vulnerability Detection

Sebastian Neef, Luca Jungnickel, Antonio Benjamin Buchholz, Valene Spence, Vicente Birke Gonzalez

View resourcePDF

Abstract

Large Language Models (LLMs) have emerged as a promising tool for automated vulnerability detection, yet their effectiveness on web-specific vulnerabilities remains to be explored. This work benchmarks six frontier (Claude Opus 4.6, Codex GPT-5.4, Gemini 3.1-pro-preview) and open-weight models (Qwen 3.5, Qwen 3 Coder Next, MiniMax M2.5) on their ability to detect real-world web vulnerabilities using static analysis in WordPress plugins, including SQL injection, stored cross-site scripting, path

Categories

Benchmarks & Evaluation

Cite

@misc{neef2026evaluating,
  title = {{Evaluating LLMs for Real-World Web Vulnerability Detection}},
  author = {Sebastian Neef and Luca Jungnickel and Antonio Benjamin Buchholz and Valene Spence and Vicente Birke Gonzalez},
  year = {2026},
  month = jun,
  eprint = {2606.21397},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.21397}
}