Why RAG Pipelines Need a Distinct Audit Approach
A retrieval-augmented generation pipeline introduces trust boundaries that do not exist in a standalone language model deployment. Data enters the system from external sources, gets transformed into vector representations, and is later retrieved and inserted directly into a prompt. Each of these transitions is a point where access control, validation, or logging can silently fail without producing an obvious error. An audit that only tests the chat interface will miss all of them.
Stage One: Data Ingestion and Source Trust
The ingestion stage determines what content ever becomes eligible for retrieval. Auditors should verify that documents are validated against an expected source list before ingestion, that file parsing does not execute embedded scripts or macros, and that ingestion pipelines record provenance (source system, timestamp, and ingesting identity) for every document. Without this record, it becomes difficult to explain why a specific piece of content was available to the model at all.
Stage Two: Embedding and Index Integrity
Embedding converts raw content into vectors, and this is the last practical checkpoint before that content becomes queryable. Confirm that sensitive fields (personal data, credentials, or internal-only classifications) are excluded or masked prior to embedding rather than filtered only at query time. Also confirm that the vector index cannot be modified by processes outside the ingestion pipeline, since an attacker who can insert vectors directly can bypass ingestion controls entirely.
Stage Three and Four: Retrieval Permissions and Prompt Construction
Retrieval is where most production gaps appear. Application-layer permissions are frequently not mirrored in the vector store, so a user's chat session may retrieve documents their account was never authorized to see. Auditors should trace a specific user identity through to the vector store query and confirm the scope matches. Once content is retrieved, prompt construction should keep system instructions, user input, and retrieved content in clearly separated regions, since retrieved text can itself contain instructions designed to override the system prompt.
Stage Five: Output Handling and Tool-Call Permissions
The final stage governs what happens after the model produces a response. If the model can trigger tool calls or actions, those calls should be subject to the same permission checks as a direct API request from the user, not granted implicitly because the request originated from the model. Output should also be logged with enough context (input, retrieved documents, and final response) to reconstruct the full request after the fact.
Governance Practices to Carry Forward From Audit Findings
An audit produces a point-in-time snapshot. The value comes from converting findings into controls that hold under normal operating load: scoped credentials enforced at query time, tool-call approval workflows, and logging that persists independent of the audit engagement. Findings that are not converted into enforced controls tend to reappear in the next audit cycle unchanged.