Indirect Prompt Injection via RAG Source (ATK-002)

Prerequisites

Target system performs Retrieval-Augmented Generation (RAG) over a corpus that includes content writable, indirectly influenceable, or sourced from semi-trusted parties (e.g., user-uploaded documents, ingested email, scraped web content, ticket descriptions, support transcripts, internal wikis with broad write access).
Retrieved context is concatenated into the model's prompt without authenticated provenance or robust sanitization.

Test approach

With written authorization, the tester seeds the RAG corpus with content that embeds adversarial instructions intended for the model rather than the human reader (for example, instructions presented in white text, in metadata fields, in trailing footnotes, or framed as supposedly system messages). The tester then issues legitimate-looking user queries that cause the seeded content to be retrieved. The objective is to determine whether the model will follow instructions originating from retrieved context — to disclose system prompt, exfiltrate sensitive data via constructed output, call a tool the user did not authorize, or produce policy-violating output. Test variations include: instructions in plain text vs. obfuscated, instructions referring to a different language, instructions only triggered by specific user phrases. The tester does not test against production data of real customers without separate explicit authorization.

Expected evidence

Seeded RAG content (saved with timestamps and hashes).
Query and full retrieved-context payload that was sent to the model.
Model response demonstrating the model followed the injected instruction.
Logs from the retrieval layer showing the seeded content was returned.
Tool-invocation logs (where applicable) showing unauthorized tool use triggered by the injection.
Cleanup record removing all seeded content from the corpus.

Detection signals

Anomalous tool invocations not present in the user's session history but triggered after a retrieval step.
Model output containing artifacts characteristic of the system prompt (length, structure, specific phrasing).
Retrieval logs showing content with high embedding similarity to known injection corpora.
Outbound network calls from agent tool layer to domains absent from a permitted-domains list, traceable back to retrieved content.

Defensive recommendations

Treat retrieved content as untrusted. Do not concatenate retrieved content into the same context window as system instructions and user input without explicit role separation (use structured prompting with clearly delimited and trust-tagged context blocks where the model API supports it).
Where possible, render retrieved content via models or layers that have no tool-calling permissions.
Apply input-side filtering at the retrieval boundary: scan retrieved chunks for known injection markers, suspicious imperative patterns, hidden text, and unusually high instruction density.
Implement provenance tagging: each retrieved chunk carries a source identifier and a trust score; trust scores below threshold cause the model to be re-prompted with explicit instruction not to follow embedded directives.
Restrict the agent's tool surface for queries whose retrieval includes low-trust sources (e.g., only read-only tools).
Continuous evaluation: include indirect injection test cases in the pre-production gate and in periodic re-tests.
Cross-reference AI-CTRL-003 (Adversarial Robustness Testing) for the program-level control.

Framework mappings

OWASP LLM Top 10: LLM01:2025
OWASP Agentic Top 10: AAI01
MITRE ATLAS: AML.T0051.001, AML.T0067
NIST AI RMF: MEASURE-2.7, MAP-5.1

Related AI Controls Catalog entries

These controls provide the audit test procedures that validate defences against this attack pattern.