Blocking prompt injection hidden in customer-uploaded KYC documents
Treat every KYC upload as untrusted input and apply layered controls at each stage: content separation, screening, least-privilege tool access, runtime policy enforcement on model interactions and tool calls, and output validation. No single control eliminates prompt injection. Retain enforcement decision records so you can show what was allowed, redacted, blocked or escalated.
Mapping layered controls to each stage
- 1
1. Upload and parsing
Screen files before the model reads them and keep extracted text labelled as untrusted data. Measure false positives on your own document population, since screening that blocks legitimate customers carries its own cost.
- 2
2. OCR and image handling
OCR output is text the model will read, so apply the same screening to it. Images sent directly to a multimodal model need their own review, because text inside an image bypasses text-layer checks.
- 3
3. Metadata and derived content
Treat metadata fields and any summary derived from a document as untrusted. Derived content does not become trusted by passing through an earlier model step.
- 4
4. Model interaction
Use content separation and spotlighting so the model can distinguish document content from instructions, and monitor for encoded content. These methods are proposed mitigations and do not give full immunity.
- 5
5. Tool calls
Grant the minimum tools per task, scope each tool (read-only versus write, specific resources) and evaluate each proposed call on operation, target, arguments and caller authority.
- 6
6. Output validation
Validate extraction output against expected structure and values. Route results that would change a verification or risk outcome to a human reviewer.
Enforcement evidence to retain for examiners and model risk review
- Timestamp and an identifier for the model interaction or tool call
- The policy and policy version that was applied
- The proposed operation, target, arguments and caller authority
- The outcome: allow, redact, block or escalate
- For escalations, the approver and the approval decision
- Only minimized, redacted content, with secrets excluded
How injection reaches model context through KYC uploads
A customer-uploaded ID, proof-of-address file or corporate registration PDF is a resource the attacker controls. NIST's adversarial machine learning taxonomy calls this capability resource control: modifying documents or pages that a model ingests at runtime. That is the precondition for indirect prompt injection.
NIST's draft guidance notes that attackers may hide injections in non-visible portions of a resource, chain them across further resources, or encode the commands. OWASP adds that an injection can affect a model even when a human cannot perceive it, as long as the model parses the content, and describes a prompt hidden in an image that accompanies benign text. FINRA describes the same pattern: instructions that are invisible or inconspicuous to people but readable by GenAI systems, with possible exposure of customer data or compromised compliance monitoring.
The practical point for a KYC pipeline is that any stage that turns a file into text or image input for a model is an entry point. Depending on your design, that includes text-layer parsing, OCR output, metadata fields and images passed to a multimodal model. Payloads also need not look like instructions. In one published resume dataset, more than 90% of hidden prompts contained no explicit instruction. That is not KYC data, but it is a reason not to depend on keyword or instruction-pattern filters alone.
Pre-ingestion file scanning is a useful layer, but it checks the file before the model reads it. It cannot reliably judge content that only becomes harmful once a model interprets it, and it does not cover content arriving through other channels.
Runtime policy enforcement on model interactions and tool calls
Stages one to three reduce what reaches the model. Stages four to six decide what the model is allowed to do with it. OWASP is explicit that text labels and prompt wording are not an enforcement boundary. Decisions should rest on the proposed operation, its target, its arguments and the caller's authority, and the model itself should be treated as an untrusted user when you test trust boundaries.
This is where enforcement in the execution path applies. Trussed enforces policy at runtime, not after the fact, on individual model interactions and tool calls, with outcomes of allow, redact, block, rate limit or escalate for human approval. In a KYC workflow, that could mean redacting identity fields from a prompt that does not need them, blocking a tool call whose target falls outside the task, or escalating a write action for approval. Scope matters: this covers each interaction and call, not an assessment of the whole document journey.
Each outcome has limits. Redaction depends on correct field detection. Blocking depends on policy that is specific enough to stop abuse without halting legitimate processing. Human approval creates a checkpoint that injection alone cannot bypass, but it adds latency and can lose value if reviewers approve by habit, so reserve it for consequential actions such as sending, deleting or modifying. Least privilege is the most effective of these controls, because it caps the damage when screening and separation both fail.
Stating residual risk and proving the controls work
NIST states that mitigations for indirect prompt injection do not offer full immunity, that no comprehensive solution exists, and that suggested defenses need further efficacy research. Model risk documentation should say so plainly: describe the layers, state the residual risk, and explain how tool permissions limit the blast radius if screening misses a payload.
Support that position with records beyond enforcement decisions. Run adversarial test suites in CI/CD whenever agent templates, tool policies or prompts change, and keep the results. Keep your false-positive measurements for document screening on your own KYC population. Enrich AI security events with AI-specific context so detection and response teams can act on them.
A sensible rollout order is to inventory every point where file content becomes model input, cut tool permissions to the minimum, add runtime enforcement on tool calls and sensitive fields, then add output validation and approval checkpoints. Trussed has run continuously in production at a regulated enterprise for 18+ months, which is relevant to how enforcement and evidence behave under live load. The control design above remains useful whichever enforcement layer you choose.
Where controls apply in the KYC document path
Ingest
Parse, extract and screen uploaded files as untrusted content.
Interpret
Separate document content from instructions before the model reads it.
Act
Enforce policy on each model interaction and tool call at runtime.
Validate
Check outputs before they influence verification decisions.
Evidence
Retain decision records that show which controls operated.
Review your KYC document path with a security architect
Discuss where runtime enforcement applies to model interactions and tool calls in your pipeline, and what evidence it produces.
Talk to an ExpertHow does your AI governance program compare?
See where your program has gaps in less than 2 minutes.