ATK-024 Tool & agent abuse
Function-Calling Schema Abuse
Applies to: Agentic AI, LLM · Last reviewed 2026-05-01
Prerequisites
- Target system uses function-calling / tool-calling APIs where the model emits structured arguments.
- Argument validation at the function layer is incomplete (e.g., trusts type but not value ranges or identifiers).
Test approach
With written authorization, the tester crafts inputs that cause the model to emit function arguments outside expected ranges: cross-tenant IDs, oversized payloads, type-confusion values, or argument values designed to exploit downstream parsers (e.g., values containing SQL or shell metacharacters).
Expected evidence
- Function-call traces with emitted arguments.
- Downstream-system logs showing the behavior the call elicited.
- Comparison to expected argument ranges.
Detection signals
- Function arguments outside historical-distribution ranges.
- Cross-tenant identifiers in function arguments.
- Arguments containing characters suspicious for downstream injection.
Defensive recommendations
- Validate function arguments at the function layer, including range and tenant-scope checks.
- Parameterize downstream queries; do not concatenate function arguments into SQL/shell.
- Reference AI-CTRL-019 (Agentic Tool Authorization).
Framework mappings
- OWASP LLM Top 10: LLM02:2025, LLM07:2025
- OWASP Agentic Top 10: AAI02
- MITRE ATLAS: AML.T0067
- NIST AI RMF: MANAGE-2.2
Related AI Controls Catalog entries
These controls provide the audit test procedures that validate defences against this attack pattern.