Implementation Guide

Implementing Runtime Controls to Prevent Enterprise AI Chatbots From Giving Tax or Legal Advice

To prevent an AI chatbot from giving legal or tax advice, enforce a written policy at runtime instead of relying on the system prompt. Define the boundary between general information and advice, classify inputs and evaluate outputs independently of the model, design refusals and escalations deliberately, log every decision, and test refusal reliability continuously.

Why system prompt instructions are not enough

A system prompt is an instruction given to the model. It is not an enforced control. If the only safeguard against tax or legal advice is a sentence in the prompt, the boundary depends entirely on the model following that sentence in every conversation.

The approach in this guide enforces a written policy at runtime, outside the model. Controls evaluate inputs and outputs independently of the system prompt, so a refusal does not depend on the model choosing to comply.

Define the boundary between information and advice

Start with a written policy that separates general information from advice. The policy is a versioned definition of permitted information and prohibited advice, and it should be approved by legal or compliance.

Every control that follows enforces this same definition, which is why the policy comes first and why its version is recorded with each decision.

A layered control architecture

Each chatbot turn passes through several layers, all driven by the same versioned policy. The figure below shows the path for a single turn.

Layered control path for a single chatbot turn

  1. PolicyA versioned definition of permitted information and prohibited advice.
  2. Input checkIntent and topic classification across the whole conversation.
  3. Output checkEvaluation of the draft response before the user sees it.
  4. Refuse or escalateA designed response or handoff when the policy is triggered.
  5. Audit logA reviewable record of each decision and the policy version used.

Classify inputs across the whole conversation

The input check classifies intent and topic across the whole conversation, not only the latest message. Conversation-level context is evaluated so that a boundary-crossing question is recognized even when it builds on earlier turns.

Evaluate outputs before the user sees them

The output check evaluates the draft response before it reaches the user, independently of the model that produced it. If a control layer fails, the result should be a safe refusal rather than an unchecked answer.

Design refusals and escalation, not just blocks

A bare block leaves the user without a path forward. Design the refusal response and the escalation handoff deliberately, so that when the policy is triggered the user receives a clear response or a defined handoff.

Every escalation should have a named owner, so that handed-off conversations are not left unassigned.

Test refusal reliability and keep the evidence

Measure refusal and false-refusal rates against a maintained test set, and retest after changes. Testing is continuous, not a one-time check at launch.

Log every blocked or escalated interaction with the policy version in force at the time. Together, test results and audit logs give reviewers a record of how the controls behaved and why.

Evaluation criteria for your implementation

Use these criteria to assess whether an implementation meets the approach described above.

  • The information versus advice boundary is written, versioned, and approved by legal or compliance.
  • Controls run on both inputs and outputs, independently of the system prompt.
  • Conversation-level context is evaluated, not only the latest message.
  • Failures in a control layer result in a safe refusal rather than an unchecked answer.
  • Refusal and false-refusal rates are measured against a maintained test set and retested after changes.
  • Every blocked or escalated interaction is logged with its policy version, and escalations have a named owner.

Enforce chatbot topic boundaries at runtime

Trussed AI provides runtime governance and security for enterprise AI agents, including runtime policy enforcement and audit logging. Talk to an expert about where these controls fit in your architecture.

Talk to an Expert