Implementation Guide

    Preventing OpenAI Structured Outputs from Returning Unmasked Account Numbers

    A schema controls the shape of a response, not the content of a string field. A field typed as a string can hold a full account number as easily as a masked one. To prevent unmasked account numbers reaching users, downstream systems and logs, combine three layers: remove or tokenize the number before the prompt, design the schema so fields carry only what the workflow needs, and enforce masking on the response at runtime. Retain evidence of each decision.

    Where unmasked account numbers enter and exit

    1. 1

      User input

      Customers and staff paste account numbers into prompts and forms. You cannot assume inputs are clean.

    2. 2

      Retrieved context

      Documents, records and search results pulled into the prompt may carry full numbers even when the user never typed one.

    3. 3

      Tool and function results

      Calls to account, billing or CRM systems often return complete records, which then become model input.

    4. 4

      Model response

      A schema-valid string field can repeat any number the model has seen.

    5. 5

      Downstream systems

      Parsed JSON flows into databases, tickets, emails and other services that may have weaker controls than the source.

    6. 6

      Logs, traces and derivative data

      Stored prompts, responses and metadata are outputs too. A masked user-facing response does not help if the full number sits in a log.

    Evidence to retain for each masking decision

    • Timestamp and the identifier of the interaction or tool call
    • Detection result: what pattern or category matched, and in which field
    • Action taken: allowed, redacted or blocked
    • Policy name and version in force at the time
    • Masked excerpt or field reference, never the raw value
    • Results and dates of seeded test runs

    What schema enforcement does and does not control

    Structured Outputs is used to get JSON that conforms to a schema you define. That is a guarantee about structure: required keys are present, types match, and the result can be parsed without repair. It is a different thing from a guarantee about meaning or content.

    Consider a field defined as a string. Any account number that appears in the prompt, in retrieved documents or in a tool result is a valid string. The model can place it in that field and the response still conforms to the schema. Conformance and disclosure are independent properties, so a response can be fully valid and still leak.

    Prompt instructions such as 'never output full account numbers' lower the probability of leakage but are probabilistic. They depend on the model following the instruction in every context, including adversarial or unusual inputs. Constraints on string fields can narrow what a field accepts, and application-side regex checks can catch known formats, but each only covers the patterns you anticipated. Treat all three as risk reducers, not as enforcement of policy.

    Layers one and two: minimize inputs and design the schema

    The most reliable control is not having the number in the model's context. Where the workflow allows, replace account numbers with tokens or truncated values before they reach the prompt, retrieved context or tool results. A model cannot repeat a value it never received. Whether truncation satisfies a given legal definition is a question for your compliance team, so decide the format against the rule text that applies to you.

    If you tokenize, check reversibility. Under Regulation P, an account number in encrypted form is not treated as an 'account number' as long as the recipient has no means to decode it. The design question is whether the model, downstream recipients or log readers can reach any mapping that turns a token back into a real number. If they can, the token offers less protection than it appears to.

    Schema design is the second layer. Define only the fields the consumer needs. If a downstream step needs the last four digits, make that the field, name it accordingly and constrain its format so that a full-length value is invalid. Avoid free-text fields such as 'notes' or 'summary' that invite the model to copy source content verbatim. Format constraints on a field, where your tooling supports them, can reject obviously wrong shapes, but they cannot tell whether a value that fits the format is permitted in context.

    Layer three: enforce masking at runtime

    Because the first two layers reduce risk without eliminating it, the response itself needs a check that sits in the path and can change the outcome. Reviewing logs afterward finds leaks that have already reached users and downstream systems. A runtime control inspects each response before it moves on and applies a defined action.

    Trussed enforces policy in the execution path across models, agents, tools and MCP servers, with allow, redact and block actions applied to individual interactions and tool calls. For this use case, a policy can detect account number patterns in a model response and redact them, or block the response entirely when the field should never contain one. Enforcement is at runtime, not reviewed after the fact. Tool call results can be handled the same way, which limits what reaches the model in the first place.

    Whichever mechanism you use, the design points are the same.

    • Define what counts as an account number: Specify full and partial values and the account types in scope, using the applicable rule text, such as the account types listed in Regulation P §1016.12.
    • Choose an action per field and context: Redact where the rest of the response is still useful. Block where any occurrence signals a failure of upstream controls.
    • Apply the same policy to inputs and outputs: Check tool results and retrieved context on the way in, and the model response on the way out.
    • Mask before storage: Apply the policy before responses and traces are written, so stored copies are masked as well.
    • Test with seeded values: Run known test account numbers through every path on a schedule and confirm each is caught.

    Regulatory context and how to apply it

    As of November 2026, Regulation P (12 CFR §1016.12) generally prohibits disclosing a consumer's credit card, deposit or transaction account number to a nonaffiliated third party for use in telemarketing, direct mail or email marketing, subject to listed exceptions. FTC guidance states the prohibition applies even if the consumer has not opted out. Whether a particular LLM provider or downstream system is such a party, and whether the use is marketing, depends on your facts and needs legal analysis.

    The FTC Safeguards Rule (16 CFR Part 314) cites account numbers as personally identifiable financial information and calls for regular testing or monitoring of key controls. That supports keeping both detection events and test results as evidence. It also supports the seeded-test approach above.

    A practical sequence: inventory entry points, remove or tokenize what the workflow does not need, tighten schemas, enforce policy on responses and tool results, then confirm that stored evidence is itself masked. Each layer covers gaps the others leave.

    Three layers for account number masking

    Before the prompt

    Minimize or tokenize account numbers in input, retrieved context and tool results.

    In the schema

    Define fields that carry only masked or truncated values where the workflow allows.

    At the response

    Enforce policy at runtime: allow, redact or block before output moves downstream.

    After the decision

    Retain detection results, actions and policy versions as evidence.

    Enforce masking where the response is produced

    See how Trussed applies allow, redact and block policy in the execution path and produces audit-ready evidence for each decision.

    Request a Demo
    Next step

    How does your AI governance program compare?

    See where your program has gaps in less than 2 minutes.