SWE-WebDevBench: Evaluating Coding Agent Application Platforms as Virtual Software Agencies

Siddhant Saxena, Nilesh Trivedi, Vinayaka Jyothi

Abstract

The emergence of "vibe coding" platforms, where users describe applications in natural language and AI agents autonomously generate full-stack software, has created a need for rigorous evaluation beyond code-level benchmarks. In order to assess them as virtual software development agencies on understanding business requirements, making architectural decisions, writing production code, handling iterative modifications, and maintaining business readiness, we introduce SWE-WebDev Bench, a 68-metric

Categories

Cite This Resource

@article{llmsec202600185,
  title = {SWE-WebDevBench: Evaluating Coding Agent Application Platforms as Virtual Software Agencies},
  author = {Siddhant Saxena and Nilesh Trivedi and Vinayaka Jyothi},
  year = {2026},
  url = {https://arxiv.org/abs/2605.04637},
}