Implementation Guide

How to Write an AI Model Card for a Fine-Tuned LLM

An enterprise model card for a fine-tuned LLM documents the base model, fine-tuning data provenance, evaluation results specific to the tuned task, intended use boundaries, and known limitations. It should be generated as part of the fine-tuning pipeline, stored where security and audit teams can access it, and detailed enough to inform access control decisions for the deployed model.

Why Fine-Tuned Models Need Their Own Documentation

A base model's original documentation describes its pretraining data, general capabilities, and known limitations at the time it was released. Fine-tuning changes model behavior in ways that base model documentation cannot anticipate. When an enterprise team fine-tunes an LLM on internal or customer-facing data, the resulting model has a different risk profile, different failure modes, and different appropriate use boundaries than the base model it started from.

Treating the base model's card as sufficient documentation for a fine-tuned derivative leaves a governance gap: security review and audit teams have no record of what data was used to change the model's behavior, why the fine-tuning was performed, or what evaluation was done on the modified system before deployment. NIST's Generative AI Profile identifies this as a value chain and component integration concern, recommending that organizations document dependencies and modifications whenever a model is adapted from a base model. A model card for a fine-tuned LLM is the artifact that closes this gap. It should be understood as a distinct document, not an addendum to the base model's card, even though it should reference the base model clearly.

Documenting Fine-Tuning Data Provenance

The most important distinction between base model documentation and fine-tuning documentation is control. An enterprise team rarely has full visibility into a foundation model's pretraining corpus, but it does control the fine-tuning dataset. This means the fine-tuning data provenance section carries more evidentiary weight for internal audit purposes than any base model disclosure.

At minimum, this section should identify where the fine-tuning data originated, whether it contains customer or regulated data, what preprocessing or filtering was applied, and who approved its use for training. If the dataset was modified across multiple fine-tuning iterations, each modification should be recorded with a timestamp and rationale, linked to the corresponding model artifact version. This version-linked history is what allows a security reviewer to reconstruct, after the fact, exactly what data shaped a given deployed model's behavior. Hugging Face's model card metadata schema supports structured, machine-readable fields for datasets and base model identity, which is useful because it allows this information to be parsed by MLOps tooling rather than left as unstructured prose that someone has to read manually during an audit.

Evaluation Results and Limitation Disclosures

Evaluation sections in fine-tuned model cards should report metrics relevant to the specific task or domain the model was tuned for, not only aggregate benchmark scores inherited from the base model. Google's original model card framework recommends disaggregating quantitative results by relevant factors, such as use case or demographic group, rather than presenting a single blended metric that can obscure uneven performance.

For a fine-tuned model, this means evaluation should be run against test cases that reflect the actual deployment scenario, and results should be captured at the same time as model validation testing. Reconstructing evaluation metrics after deployment, once the original test conditions are no longer available, produces weaker documentation and should be avoided.

Limitation disclosures should be specific rather than generic. Stating that a model "may produce inaccurate outputs" provides no operational value. Documenting that a model was fine-tuned on a narrow set of customer support transcripts and has not been evaluated on legal or medical queries gives downstream teams something they can act on.

Defining Intended Use and Out-of-Scope Boundaries

Intended use sections are often written as boilerplate, describing a model's purpose in broad terms without enough specificity to inform any downstream decision. For governance purposes, this section needs to do more work. It should state the specific task the model was fine-tuned to perform, the user population it was evaluated against, and the conditions under which it was validated.

The out-of-scope section is equally important and frequently underwritten. It should list use cases the model was not evaluated for and should not be applied to, based on the limitations of the fine-tuning data and evaluation coverage. This level of specificity matters beyond documentation completeness: the intended use and out-of-scope sections are the basis on which an organization can define what the deployed model should and should not be permitted to do at runtime. A model card that clearly states a model was fine-tuned for internal document summarization and was not evaluated for financial advice gives an access control or policy team concrete boundaries to enforce, rather than requiring them to infer boundaries from the model's general capabilities.

Regulatory Context and Its Practical Limits

The EU AI Act establishes documentation obligations for providers of general-purpose AI models, requiring technical documentation of training data sources and model capabilities, along with information sharing obligations toward downstream providers integrating the model into their own systems. These obligations take effect in phases and apply primarily to providers of general-purpose models, so their direct applicability to an enterprise fine-tuning a third-party model for internal use may vary depending on the organization's role in the value chain.

Regardless of specific regulatory applicability, the underlying documentation practices (data provenance, capability disclosure, and known limitations) align closely with what NIST's AI RMF describes under its "map" and "measure" functions. Model cards should be built as part of this broader risk management structure rather than as a standalone compliance artifact, since the same documentation that supports a regulatory obligation also supports internal security review, incident investigation, and deployment decision-making.

Practical Guidelines for Model Card Quality

  • Write intended use and out-of-scope sections specifically enough to inform an access control decision, not as generic boilerplate.
  • Separate base model documentation from fine-tuning-specific documentation rather than merging them into one ambiguous section.
  • Disaggregate evaluation results by use case where feasible, rather than reporting a single aggregate score.
  • Version model cards alongside model artifacts so any deployed version has a matching, accurate card.
  • Store completed cards in a location accessible to security and audit teams, not only ML engineering.

Core Sections of a Fine-Tuned LLM Model Card

Each section below addresses a distinct governance need, from establishing lineage to defining enforceable runtime boundaries.

Base Model Identity

Name, version, and source of the pretrained model being fine-tuned.

Fine-Tuning Data Provenance

Dataset sources, licensing, and modification history specific to the tuning process.

Task-Specific Evaluation

Metrics measured against the fine-tuned use case, not just general benchmarks.

Intended Use and Boundaries

Specific approved use cases and explicit out-of-scope scenarios.

Known Limitations

Disclosed failure modes and conditions where output reliability degrades.

Turn Model Card Documentation Into Enforceable Runtime Boundaries

A well-documented model card defines what a fine-tuned model is meant to do. Trussed AI helps enterprise teams enforce those boundaries at runtime through policy enforcement, permissioning, and monitoring for deployed AI agents.

Explore Runtime Governance