Adversarial Robustness Testing for Predictive ML and Computer Vision

Objective

Validate that production predictive ML, computer-vision, recommender, and speech models have been tested against evasion attacks, model-extraction attacks, membership-inference attacks, data-poisoning detection, and (for vision) physical-world perturbations, with documented findings and remediation.

Applicability

AI types

Traditional ML, Computer Vision, Recommender, Speech

Deployment models

Self-hosted, Hybrid, Edge, Embedded

Lifecycle stages

Evaluation & Testing, Operation & Monitoring

Risk domains

Model

Regulatory regimes

EU AI Act, ISO 42001, NIST AI RMF, Banking, Healthcare, Public sector

Company size

MidMarket, Enterprise

Rationale

Companion control to AI-CTRL-003 (which addresses LLM-specific adversarial testing). Predictive ML, computer-vision, recommender, and speech models face different attack surfaces: evasion (adversarial examples), extraction (querying for surrogate models), membership inference (privacy attack), and poisoning. ISO/IEC 42001 clause 9.1 and EU AI Act Article 15 require robustness testing appropriate to the system type. Without dedicated testing using methods appropriate to the model class, robustness claims are unverified.

Control narrative

Each in-scope predictive ML, CV, recommender, or speech system in production undergoes adversarial robustness testing pre-production, on material change, and at least annually. Testing scope is tailored to system type: for CV — adversarial perturbations (FGSM, PGD), physical-world perturbations where applicable (printable adversarial patches, lighting/distance variation); for predictive ML — evasion robustness on protected-feature manipulations and label-noise/distribution-shift robustness; for recommender systems — manipulation robustness (review/click bombing); for speech — adversarial audio. For all classes — model-extraction and membership-inference resistance is evaluated under realistic query budgets. Findings flow to the standard findings register with severity, remediation owner, target date, and re-test verification.

Test of Design

Procedures

1. Obtain the Adversarial Testing standard and confirm separate scope for non-LLM AI types complementing AI-CTRL-003.
2. Confirm test-method requirements by system type (CV, predictive ML, recommender, speech).
3. Confirm milestones (pre-production, material change, ≥ annual).
4. Confirm physical-world testing scope for safety-relevant CV systems (e.g., autonomous, medical imaging).
5. Confirm linkage to AI-CTRL-003 to avoid scope gaps and overlaps.

Inquiries

• Who selects appropriate attack methods for each system type?
• How is realistic query budget determined for extraction / membership inference testing?
• How are CV physical-world tests conducted ethically and safely?
• How is the test corpus refreshed as new attack research emerges?

Inspections

• Adversarial Testing standard (non-LLM scope).
• Approved attack-method catalogs by system type.
• Sample test reports.
• Findings register.

Test of Operating Effectiveness

Procedures

1. From the inventory, sample in-scope production systems by type.
2. Confirm pre-production test occurred prior to deployment, material-change tests for each change in period, and at least annual test in period.
3. Inspect test reports and confirm methods used match the standard's requirements for the system type.
4. Confirm findings are tracked with severity, owner, target date, re-test.
5. Re-perform 1 test method on a non-production replica and compare results.

Sample-size guidance

Reperformance

• For 1 sampled CV system, independently execute a PGD adversarial attack on the documented evaluation set and compare success rate to management's report.

Evidence requirements

Required

• Adversarial Testing standard (non-LLM) PDF/Word · At fieldwork
• Test reports for sampled systems PDF/Notebook · Per sample
• Findings register Excel/system · At fieldwork
• Re-test evidence for closed findings Logs / report · Per sample

Supporting

• Attack-method catalogs PDF/repo · Annual

Retention: 7 years for regulated environments; 3 years otherwise

Framework mappings

Related controls

• AI-CTRL-003

  Adversarial Robustness Testing for LLM Systems

• AI-CTRL-010

  Model Versioning and Change Control

• AI-CTRL-017

  Pre-Production AI Evaluation Gates

References

• NIST AI 100-2e2023 — Adversarial Machine Learning
• MITRE ATLAS
• EU AI Act Article 15