Most enterprises now run a mix of third-party foundation models, fine-tuned variants, and internally trained models across different teams and environments. Few of these have a standardized record explaining where the model came from, what data shaped it, who modified it, and where it has been deployed. This is not a hypothetical gap. When a governance team needs to answer a basic question, such as which production systems depend on a specific model version, or whether a model's training data included restricted content, the answer often lives in scattered documentation, chat threads, or the memory of the engineer who built it. An AI model passport addresses this by defining a consistent, structured record that travels with the model across its lifecycle, from initial training or acquisition through fine-tuning, deployment, and eventual retirement.
What Provenance Records Typically Capture
While no single universal schema governs AI model passports today, the concept parallels a software bill of materials extended to model-specific attributes. In practice, a provenance record generally needs to capture:
- A persistent identifier for the specific model version
- Information about training data sources and any known lineage constraints
- A history of modifications, such as fine-tuning or parameter changes
- The identity of the party responsible for each change
- A record of where and how the model has been deployed
The exact field set an organization adopts should be defined deliberately by governance policy rather than assumed, and validated against whatever named standards or regulatory expectations apply to that organization's sector and jurisdiction.
Provenance as Input to Runtime Governance, Not a Static Document
The most common mistake is treating a model passport as a document produced once at deployment and filed away for audits. Framed this way, it becomes stale the moment a model is retrained or fine-tuned, and it provides little operational value.
A more useful framing treats provenance data as a machine-readable input to runtime governance systems. If a model's identity, version, and origin are known and verifiable, that information can inform policy decisions at runtime, such as what tools or data an agent built on that model is permitted to access, what actions require approval, and what should be logged for later review. This connects provenance documentation directly to practices like agent identity verification and least-privilege enforcement, where knowing what a model is and where it came from is a precondition for deciding what it should be allowed to do.
The Third-Party Model Problem
Internally developed models are the easier case, since the organization controls the training process and can document it directly. Third-party and open-source models introduce a harder problem: the deploying organization often has no direct visibility into training data lineage or modification history, and must rely on whatever the vendor discloses.
Governance policy needs to explicitly address this gap rather than assume parity between internal and external models. A defined process should specify:
- What minimum provenance information is required before a third-party model is approved for production use
- Who has authority to accept residual risk when that information is incomplete
- How that risk acceptance is documented so it can be revisited later, particularly if the model is later implicated in an incident
Evaluation Criteria for a Provenance Program
- Define minimum required provenance fields before any model, internal or third-party, is approved for production
- Assign a named organizational owner for passport accuracy and updates
- Require passport data to be structured for automated policy engine consumption, not just human review
- Build passport updates into existing model change and deployment workflows rather than as a separate process
- Define an explicit escalation path for models with incomplete or unverifiable provenance
- Treat provenance documentation as an input to risk assessment, not a standalone compliance deliverable