ATK-021 Model integrity
Supply Chain — Compromised Model Hub
Applies to: LLM, Traditional ML, Computer Vision, Multi-modal · Last reviewed 2026-05-01
Prerequisites
- Target organization downloads models from third-party model hubs (e.g., Hugging Face, internal model registries with weak provenance).
Test approach
Defensive assessment: the tester reviews provenance, signing, and verification controls for model downloads. The tester does not upload trojaned models to public hubs. Where authorized, the tester simulates ingestion of a known-test-trojaned model from a sanctioned internal mirror and observes whether detection controls trigger.
Expected evidence
- Configuration evidence for model verification at ingestion (signing, hash check).
- Simulated-trojan ingestion logs.
- SBOM-equivalent records for models in production.
Detection signals
- New model versions downloaded without corresponding verification logs.
- Models from unknown publishers in production.
- Anomalous behavior post-deployment (drift, unusual output patterns) that may indicate a backdoored model.
Defensive recommendations
- Maintain a model SBOM with publisher, version, hash, and verification status.
- Require signing or hash verification before model deployment.
- Pin model versions in production; do not auto-pull latest.
- Reference AI-CTRL-004 (Vendor Due Diligence) and AI-CTRL-010 (Change Control).
Framework mappings
- OWASP LLM Top 10: LLM05:2025
- MITRE ATLAS: AML.T0010
- NIST AI RMF: MAP-4.1
Related AI Controls Catalog entries
These controls provide the audit test procedures that validate defences against this attack pattern.