ATK-010 Output handling
Insecure Output Handling — Code Execution
Applies to: LLM, Agentic AI · Last reviewed 2026-05-01
Prerequisites
- Downstream system executes code emitted by the model (e.g., generated SQL run against a database, generated shell commands run by an agent, generated Python evaluated in a sandbox).
- Lack of strong sandboxing or input parameterization between model output and execution.
Test approach
With written authorization in a non-production environment, the tester crafts prompts that cause the model to emit code containing injection payloads (SQL injection, command injection, prototype pollution, deserialization), and observes whether the downstream executor invokes the payload. The tester does not exercise these attacks against production infrastructure.
Expected evidence
- Prompts that elicited malicious code generation.
- Generated code containing the payloads.
- Logs from the executor showing execution (or block).
- Sandbox isolation evidence.
Detection signals
- Generated code containing untrusted-style SQL fragments, shell metacharacters, deserialization markers.
- Executor logs showing commands not derivable from user intent.
- Network egress from the executor to unexpected destinations.
Defensive recommendations
- Treat model output as untrusted; parameterize all generated SQL, never pass to shell directly.
- Sandboxed execution with strict resource and network limits for any generated code execution path.
- Static analysis or LLM-based pre-execution review for high-impact actions.
- Cross-reference AI-CTRL-007 and AI-CTRL-019.
Framework mappings
- OWASP LLM Top 10: LLM02:2025
- OWASP Agentic Top 10: AAI02
- MITRE ATLAS: AML.T0050
- NIST AI RMF: MEASURE-2.7, MANAGE-2.2
Related AI Controls Catalog entries
These controls provide the audit test procedures that validate defences against this attack pattern.