Personal Data Minimization in Training and Inference

Objective

Ensure that personal data used in training, fine-tuning, evaluation, and inference is minimized to the data strictly necessary for the stated purpose, processed under a defensible lawful basis, and subject to the rights of affected individuals.

Applicability

AI types

LLM, Agentic AI, Traditional ML, Computer Vision, Generative AI, Recommender

Deployment models

SaaS, Self-hosted, Hybrid, Edge

Lifecycle stages

Data Sourcing & Preparation, Training, Deployment, Operation & Monitoring

Risk domains

Data

Regulatory regimes

GDPR, PIPEDA, HIPAA, EU AI Act, State privacy laws

Company size

SMB, MidMarket, Enterprise

Rationale

Privacy regulators worldwide apply data-minimization principles (GDPR Article 5(1)(c), PIPEDA Schedule 1 §4.4, HIPAA Privacy Rule). Training and inference both involve personal data: training datasets, fine-tuning examples, RLHF feedback, conversation logs used for improvement, RAG retrieval over personal-data corpora, embeddings derived from personal data. Without minimization, organizations create unnecessary privacy exposure, complicate data-subject-rights responses, and violate principles regulators are actively examining.

Control narrative

For every AI system processing personal data, the organization documents the lawful basis, identifies the minimum data categories required, applies pseudonymization or de-identification where possible, restricts retention of inference data (prompts, completions, embeddings) to the minimum necessary, and supports data-subject rights (access, erasure, rectification, restriction). For training datasets, personal data is reviewed for minimization at curation; for inference, prompts and completions are not retained beyond operational necessity unless an explicit, lawful, documented purpose applies. Default settings on SaaS AI vendors are configured to opt out of training-on-customer-data. The control integrates with the organization's broader privacy program (DPIA, ROPA, DSR workflow).

Test of Design

Procedures

1. Obtain the AI Personal Data standard and confirm scope, minimization principles, and integration with the broader privacy program.
2. Confirm requirement to document lawful basis per AI system and per data category.
3. Confirm requirement to opt out of vendor training-on-customer-data by default.
4. Confirm retention defaults for prompts, completions, and embeddings.
5. Confirm support for data-subject rights including erasure where applicable to training data and embeddings.

Inquiries

• How is the minimum-necessary determination made and documented?
• How are erasure requests handled for data embedded in trained models (i.e., when retraining is impractical)?
• How are vendor defaults verified and re-verified periodically?
• How is personal data identified in unstructured RAG sources?

Inspections

• AI Personal Data standard.
• DPIAs / Privacy Impact Assessments for sampled systems.
• Vendor configuration evidence (opt-out of training-on-customer-data).
• Retention configuration evidence.

Test of Operating Effectiveness

Procedures

1. From the inventory, sample AI systems processing personal data.
2. For each, confirm a current DPIA exists, lawful basis is documented, and minimization rationale is recorded.
3. Confirm SaaS vendors are configured to opt out of training-on-customer-data (capture configuration evidence).
4. Confirm retention configuration matches the documented standard.
5. For sampled data-subject-rights requests involving AI systems, confirm requests were handled per the documented workflow including retraining/un-embedding where applicable.
6. Sample inference logs and confirm content retention does not exceed the documented period.

Sample-size guidance

Reperformance

• For 1 sampled vendor, independently log into the admin console and confirm the opt-out toggle is enabled.

Evidence requirements

Required

• AI Personal Data standard PDF/Word · At fieldwork
• DPIAs for sampled systems PDF/Word · Per sample
• Vendor opt-out configuration evidence Screenshot / config export · Per sample
• Retention configuration evidence Config screenshot · Per sample

Supporting

• Data-subject-rights handling records involving AI systems Ticket export · Per sample

Retention: Lifetime of system + per applicable privacy law (typically 6 years post-relationship for GDPR-context)

Framework mappings

Related controls

• AI-CTRL-002

  Training Data Provenance and Lineage

• AI-CTRL-007

  Output Filtering and Content Moderation

• AI-CTRL-015

  Data Retention and Deletion for AI Systems

References

• GDPR Article 5 — Principles relating to processing of personal data
• EDPB Opinion 28/2024 — AI models and personal data
• OWASP GenAI Data Security 2026 — Personal data minimization