Model Cards and Datasheets: What Regulators Now Expect to See

From Academic Format to Regulatory Requirement

Model cards and datasheets originated as academic proposals. Model Cards for Model Reporting (Mitchell et al., 2019) described a structured format covering intended use, training and evaluation data, and performance across demographic groups. Datasheets for Datasets (Gebru et al., 2018 and 2021) proposed similar structure for dataset documentation, covering motivation, composition, and collection process.

These formats were designed for static, single-release disclosure, typically published alongside a model or dataset at the time it was made public. That design assumption is now the primary point of friction as regulators formalize documentation requirements for enterprise-deployed systems, which are updated, fine-tuned, and retrained continuously in production rather than released once.

The Two Anchor Frameworks Enterprises Are Measured Against

EU AI Act (Regulation (EU) 2024/1689)

The EU AI Act requires providers of high-risk AI systems to maintain technical documentation under Article 11, with required content specified in Annex IV. That content includes a description of the system's intended purpose, its design specifications, and information on training, validation, and testing datasets, including their provenance. Documentation obligations are tied to a system's risk classification under Annex III, meaning governance teams must first establish and record the risk tier before determining documentation scope.

NIST AI Risk Management Framework and AI 600-1

NIST's AI Risk Management Framework (AI 100-1, January 2023) organizes governance activity into four functions: Govern, Map, Measure, and Manage. It treats documentation as a mechanism for risk traceability across the AI lifecycle rather than a standalone deliverable. NIST's Generative AI Profile (AI 600-1, July 2024) adds a documentation category not present in earlier academic formats: tracking content provenance for generative outputs.

Where Traditional Model Cards Fall Short

Academic-style model cards and datasheets remain structurally useful, but three gaps consistently appear when they are mapped against current regulatory content categories.

• Data lineage: Annex IV and Datasheets for Datasets both call for traceability back to original data sources and collection processes, which many existing templates document only at a summary level.
• Validation testing detail: Annex IV expects documentation of testing evidence, not just a description of intended use.
• Version specificity: A model card written for a base model does not necessarily describe the behavior of a fine-tuned or policy-adjusted version running in production.

Multi-agent or chained-model deployments compound this gap. A single model card format was not built to capture composite system behavior across multiple models or agents interacting in a pipeline, which is increasingly how enterprise AI systems are architected.

Static Documentation Versus Ongoing Operational Evidence

Both frameworks treat documentation as part of an ongoing risk management process rather than a one-time compliance artifact. This is a meaningful shift from the original academic intent. A static model card produced at release describes the system as it existed at one point in time. Regulators reviewing high-risk systems, and NIST's lifecycle-oriented guidance, expect evidence that documentation reflects the system's current, deployed state.

This creates a practical requirement: documentation update processes should be triggered by model retraining, fine-tuning, or prompt and policy changes, not only at initial release. Runtime logging of policy enforcement and model behavior can serve as supplementary evidence alongside static documentation, helping demonstrate how a system actually behaved in production over time.

Building an Audit-Ready Documentation Process

Moving from a static model card practice to one that satisfies regulatory scrutiny requires connecting documentation workflows to the AI system lifecycle. Key process considerations include:

• Mapping each deployed system to the appropriate EU AI Act Annex III risk tier before scoping documentation requirements.
• Maintaining version-specific records for each fine-tuned or policy-updated model variant, not only the base model release.
• Defining explicit triggers that initiate a documentation update, such as retraining events, serving policy changes, or dataset updates.
• Capturing runtime logs and policy enforcement records that can supplement static documentation during a regulatory audit.
• Assigning clear documentation ownership across data science, legal, and governance functions, with accountability enforced operationally rather than assumed.

Questions Governance Leaders Should Be Able to Answer

• Does our model card or datasheet template cover every content category specified in EU AI Act Annex IV, including data provenance and validation testing?
• Can we produce documentation reflecting the specific deployed version of a model or agent at a given point in time, not just its base version?
• What process triggers a documentation update when a model is fine-tuned, retrained, or its serving policy changes?
• Do we have operational evidence, such as logs, that can supplement static documentation during an audit?
• Who owns documentation accuracy across data science, legal, and governance functions, and how is that ownership enforced day to day?

Frequently Asked Questions