Model risk

Validate that production-bound LLM and agentic AI systems have been tested against direct prompt injection, indirect prompt injection, jailbreak, refusal evasion, and (where applicable) multi-modal adversarial inputs, with documented findings, remediation, and re-test cycles.

Ensure that third-party AI products and AI-enabled services are subject to risk-based due diligence covering data handling, model provenance, security testing, incident response, compliance posture, and contractual safeguards before procurement and on an ongoing basis.

Ensure that each AI system is subject to a documented risk assessment and AI System Impact Assessment (AIIA) aligned with ISO/IEC 23894 and ISO/IEC 42005, completed prior to production deployment and refreshed on material change or annually.

Apply layered output filtering and content moderation to LLM and generative AI systems to prevent disclosure of sensitive data, prohibited content, executable payloads, and policy-violating outputs.

Ensure that AI systems produce logs sufficient to detect abuse, investigate incidents, demonstrate compliance, and reconstruct decisions, with logs protected from tampering and retained per regulatory requirements.

Maintain a tested AI-specific incident response capability that detects, triages, contains, and recovers from AI incidents (prompt injection, model misbehavior, data exfiltration, decision errors with material impact), with regulator notification per applicable rules.

Ensure that every model promoted to production is uniquely versioned, traceable to its training data, evaluation results, and approval, and subject to controlled change-management with rollback capability.

Validate performance, fairness, and disparate-impact metrics for AI systems with protected-class implications, using methods appropriate to the system type and use case, with documented findings and remediation.

Maintain current, accessible technical documentation (model cards, datasheets, system cards) for every production AI system, sufficient to enable risk-based use, regulatory review, and downstream consumer understanding.

Validate that production predictive ML, computer-vision, recommender, and speech models have been tested against evasion attacks, model-extraction attacks, membership-inference attacks, data-poisoning detection, and (for vision) physical-world perturbations, with documented findings and remediation.

Require explicit, documented evaluation against defined thresholds — covering performance, fairness, robustness, safety, and (where applicable) drift baselines — as a gate to production deployment for every AI system.

Ensure that AI system decommissioning follows a documented procedure addressing dependent system continuity, data retention/deletion, model artifact handling, vendor termination assistance, documentation preservation, and stakeholder communication.

Restrict agentic AI systems to least-privilege tool authorizations; enforce approval boundaries for high-impact actions; log and review all tool invocations; and prevent privilege escalation across multi-step or multi-agent workflows.