AI System Logging and Monitoring

Objective

Ensure that AI systems produce logs sufficient to detect abuse, investigate incidents, demonstrate compliance, and reconstruct decisions, with logs protected from tampering and retained per regulatory requirements.

Applicability

AI types

LLM, Agentic AI, Traditional ML, Computer Vision, Generative AI, Multi-modal

Deployment models

SaaS, Self-hosted, Hybrid, Edge

Lifecycle stages

Deployment, Operation & Monitoring

Risk domains

Infrastructure, Model

Regulatory regimes

EU AI Act, ISO 42001, NIST AI RMF, Banking, Healthcare

Company size

MidMarket, Enterprise

Rationale

EU AI Act Article 12 mandates automatic logging for high-risk systems. ISO/IEC 42001 clause 8.4 and NIST AI RMF MANAGE-4 require traceability and explainability sufficient to support after-the-fact review. Without sufficient logging, abuse detection (prompt injection, agent misuse, data exfiltration), incident investigation, and contestability of decisions are impossible.

Control narrative

Each AI system within scope generates logs covering: invocation metadata (timestamp, user, session, source IP / device, model version, prompt token count, completion token count, tool calls, retrieved sources for RAG, latency), authentication and authorization events, configuration changes, prompt and completion fingerprints (or full content where lawful and necessary, with separate access controls), policy/guardrail events (block/warn/allow with rule ID), and tool/agent action records. Logs are forwarded to the central SIEM, time-synchronized, tamper-protected, and retained per the data classification (minimum 1 year operational; 7 years for regulated). Sensitive content in logs is access-controlled separately from operational metadata. Detection rules covering prompt injection signals, anomalous tool use, and known abuse patterns are deployed.

Test of Design

Procedures

1. Obtain the AI Logging standard and confirm coverage of required event categories.
2. Confirm log retention is specified by data classification and regulatory regime.
3. Confirm tamper protection (write-once, integrity hashing) and access controls on sensitive log content.
4. Confirm forwarding to central SIEM with time synchronization.
5. Confirm detection rule coverage for known AI abuse patterns (prompt injection, anomalous tool use, sensitive-data egress).

Inquiries

• Who owns AI logging design?
• How is sensitive content in logs (e.g., user prompts containing PII) handled?
• How are logs from third-party AI services (SaaS) ingested?
• What is the detection content development cadence?

Inspections

• AI Logging standard.
• Log schemas for sampled systems.
• SIEM ingestion confirmation.
• Detection rule inventory.

Test of Operating Effectiveness

Procedures

1. From the inventory, sample production AI systems.
2. For each, inspect log samples and confirm presence of required event categories.
3. Confirm log retention configuration matches the data classification.
4. Test integrity protection by confirming write-once or hash-chain implementation.
5. Confirm detection rules are active in the SIEM and have fired (or are tuned reasonably for FPs) in the audit period.
6. Trace 3 logged events through to SIEM ingestion and confirm time synchronization.

Sample-size guidance

Evidence requirements

Required

• AI Logging standard PDF/Word · At fieldwork
• Log schemas / field dictionaries per sampled system PDF/CSV · Per sample
• Log samples Log export · Per sample
• SIEM detection rule inventory System export · At fieldwork

Supporting

• Integrity / tamper-protection configuration Config screenshot · At fieldwork

Retention: Minimum 1 year operational; 7 years for regulated content; 10 years for EU AI Act high-risk

Framework mappings

Related controls

• AI-CTRL-003

  Adversarial Robustness Testing for LLM Systems

• AI-CTRL-009

  AI Incident Response Procedures

• AI-CTRL-020

  Shadow AI Detection

References

• EU AI Act Article 12 — Record-Keeping
• ISO/IEC 42001:2023, clause 8.4
• NIST AI RMF — MANAGE-4 Continuous Monitoring