Supervising LLM-Generated Research Summaries at a Broker-Dealer
Treat each LLM-generated research summary as a firm communication inside your existing supervisory system. FINRA says firms are responsible for their communications whether a human or AI generates them, and that its rules are technology neutral. In practice, classify each use case by audience, restrict the sources the model may use, check outputs before delivery, route defined content classes to a principal, and keep evidence for every interaction.
- 1
Rule 3110, supervision
Assign supervisory ownership for the summarization tool as for any other communication channel, and reflect the tool and its controls in written supervisory procedures (WSPs).
- 2
Rule 2210, communications
Firms must ensure AI-generated communications they distribute or make available comply. Audience determines whether pre-use principal approval applies or whether procedure-based supervision is permitted for institutional communications.
- 3
Rule 2241, research
Conflicts policies and required disclosures apply to research reports. Review whether a summary can omit or alter that content. Rule 2241(h)(1) review of third-party research depends on the independent third-party and institutional communication exceptions.
- 4
Exchange Act Rule 17a-4(b)(4)
Communications relating to the firm's business as such must be retained for at least three years, the first two in an easily accessible place. Delivered summaries that are communications fall in scope.
- 5
Customer information
FINRA's AI report points to Regulation S-P policies for protecting customer information and records. Check what customer data can reach the model.
- 6
1. Input and source restriction
Define which reports, notes and commentary the tool may read. Exclude restricted or confidential material, and block requests that reach outside the approved scope.
- 7
2. Output policy check
Place a check between the model output and the reader. Redact restricted or confidential content, and block outputs that fail defined policy tests.
- 8
3. Human-approval escalation
Route defined content classes to a principal or supervisory analyst before delivery. Candidates include summaries of third-party research outside the institutional exceptions, and retail-facing content.
- 9
4. Evidence retention
Write a record of the decision, inputs and outcome for every interaction, retained under the schedule counsel sets.
- Update WSPs to describe the AI tool, its supervisory controls and escalation paths, then test the updated procedures as FINRA advises
- Assign a named supervisory owner for each summarization use case
- Write approval rules by audience and research origin, citing the Rule 2210(b)(3) and Rule 2241(h)(1) review points
- Set the retention schedule with counsel, covering both delivered summaries and any prompts and logs treated as records
- Run an examiner-style exercise: pick a delivered summary and reconstruct its sources, policy decision and any approval
Evidence a supervisory program should be able to produce
- The policy version in force when the summary was generated, showing what the tool was permitted to do
- The prompt and the source documents that produced the summary
- The generated output and what was delivered, including any redaction applied
- The policy decision (allow, redact, block or escalate) and the rule that triggered it
- The approver, timestamp and disposition for each escalated summary
- Retention records showing period, accessible form and who can retrieve them
Start from existing obligations, not a new category
FINRA Regulatory Notice 24-09 states that FINRA rules are technology neutral and apply to generative AI as they do to any other technology. The Notice creates no new requirements. It also names, as an example of generative AI use, generating summaries derived from a member firm's research reports, and obtaining issuer information from SEC filings and earnings call transcripts. A research summarization tool is therefore squarely within the use cases FINRA has described.
The practical consequence is that the summary is your communication. Rule 3110 requires a supervisory system reasonably designed to achieve compliance for each associated person's activities, and FINRA says this applies regardless of the technology used. Firms that supervise AI applications have told FINRA they want to understand how the applications function, how outputs are derived, and whether actions taken on those outputs align with legal and compliance requirements. That is the question an examiner is likely to ask of your program.
Classify each use case before designing controls
Inventory every place an LLM summarizes research, analyst notes or market commentary. For each, record the audience (internal, institutional, retail or other) and the origin of the underlying material. Audience drives which Rule 2210 path applies, and origin matters because the firm's own research and third-party research carry different review obligations under Rule 2241.
Keep third-party and proprietary research separate in the workflow, with distinct source scopes and distinct approval rules. A single undifferentiated corpus makes it hard to show which review path a given summary followed.
Two questions need a decision from counsel rather than a technical default. First, which summaries are communications under Rule 2210 and which qualify as institutional communications. Second, whether prompts, outputs and policy records are themselves retained communications under Rule 17a-4(b)(4). The answers set the retention design and the escalation thresholds.
Runtime enforcement and after-the-fact review are different controls
Sampling and lookback review examine a summary after it has reached a reader. They can find patterns, but they cannot show that a restricted passage was withheld from a specific summary, and they depend on records that may not capture the prompt, the sources or the policy applied. Controls enforced at runtime act before delivery, and they leave a record of each decision as a by-product of enforcement.
This is a design distinction, not a regulatory preference. Runtime enforcement complements principal review and model risk management and does not replace either. Principals still approve the content classes your procedures assign to them. Model risk management, which FINRA names alongside data governance, customer privacy and supervisory control systems as adoption factors, still assesses whether the tool is fit for the use case. Runtime enforcement supplies the interaction-level evidence that connects those functions to what the tool actually did.
Trussed enforces policy in the execution path, with allow, redact, block, rate limit and escalate-for-human-approval actions on individual interactions and tool calls. It generates audit-ready evidence mapped to frameworks including NAIC, NYDFS, NIST and the EU AI Act, and deploys self-managed in the customer's cloud with configurable retention and data residency. Insurance, a regulated financial sector, is where Trussed is deepest.
From obligation to interaction-level control
Obligation
Rules 3110, 2210, 2241 and 17a-4(b)(4) apply to the summary.
Control point
Sources, output checks, approval routing, retention.
Runtime action
Allow, redact, block or escalate each interaction.
Evidence
A record per interaction that an examiner can follow.
Test your supervision design against a real summary
Talk to an expert about interaction-level enforcement and the evidence a supervisory program needs to produce.
Talk to an ExpertHow does your AI governance program compare?
See where your program has gaps in less than 2 minutes.