paperAugust 2026Unreviewed

Security-First Evaluation of Text-to-Terraform: Benchmarking LLMs and SLMs for Secure IaC Generation

Francis Luis Santos Vargas, Rodrigo Brandão Mansilha, Diego Kreutz

Abstract

Cloud misconfiguration remains a leading cause of security incidents, yet whether LLMs and SLMs can generate security-compliant Infrastructure-as-Code is an open question. We benchmark seven models, three closed LLMs (Claude Opus 4, GPT-5.4, Gemini 2.5 Pro) and four open SLMs (Qwen2.5-Coder-14B, WizardCoder-33B, CodeLlama-13B, Magicoder-S-CL-7B), on AWS Terraform generation across 17 scenarios, integrating Checkov and Trivy scanners into a GitLab CI/CD pipeline and evaluating two prompt strategi

Cite

@misc{vargas2026securityfirst,
  title = {{Security-First Evaluation of Text-to-Terraform: Benchmarking LLMs and SLMs for Secure IaC Generation}},
  author = {Francis Luis Santos Vargas and Rodrigo Brandão Mansilha and Diego Kreutz},
  year = {2026},
  month = aug,
  eprint = {2608.02672},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.02672}
}