Databricks AI Governance: Unity Catalog and Runtime Controls
How Unity Catalog and Databricks runtime controls govern AI agents, where their responsibilities diverge, and where additional runtime enforcement is needed once compute is provisioned.
Unity Catalog governs who and what can access data, models, and registered functions across Databricks workspaces attached to a metastore, using GRANT-based permissions, lineage tracking, and a catalog-schema-object namespace. Databricks runtime controls, including cluster policies, compute access modes, and workspace permissions, govern how compute executes and which Unity Catalog enforcement features are active for a given workload. For AI agents that call Unity Catalog-registered functions as tools, data-access governance and execution-time behavior are enforced through different mechanisms, and native controls alone may not fully govern an agent's behavior once compute is provisioned.
Two governance layers, one workload
Enterprise AI on Databricks typically depends on two separate control planes. Unity Catalog governs access to data and assets. Runtime controls govern how compute is configured and which enforcement features apply when a workload runs. Understanding both layers, and the boundary between them, is essential when AI agents call registered functions as tools.
Unity Catalog
Governs access to tables, models, functions, and lineage across a metastore.
Cluster policies
Restrict compute configuration, instance types, and allowed libraries.
Compute access modes
Determine whether Unity Catalog fine-grained ACLs are enforced at runtime.
AI Gateway / Model Serving
Apply request-level controls to deployed model endpoints.
Unity Catalog: the data and asset governance layer
Unity Catalog is Databricks' centralized governance layer for data, models, functions, and related assets. It organizes objects into a three-level namespace of catalog, schema, and object, and applies this structure consistently across every workspace attached to the same metastore. Access is controlled through GRANT and REVOKE statements, including row-level and column-level security policies that apply uniformly regardless of which workspace a query originates from.
Unity Catalog also automatically captures data lineage at the column and table level across queries, notebooks, jobs, and dashboards, without requiring manual configuration. That gives platform teams a built-in record of how data moves through pipelines and into models.
For AI workloads specifically, this governance extends to registered ML models and to functions that can be exposed as callable tools for AI agents. Tool access is governed under the same permission model applied to tables and views.
Runtime controls: governing how compute executes
Where Unity Catalog governs what can be accessed, Databricks runtime controls govern how compute is provisioned and executed. Cluster policies allow administrators to restrict configuration options such as instance types, autoscaling limits, allowed libraries, and Spark configuration, preventing users from provisioning compute outside approved parameters.
Compute access modes, particularly the choice between single-user and shared clusters, determine process isolation and directly affect which Unity Catalog governance features are enforced at runtime. Fine-grained access control depends on the access mode selected for a given cluster.
Workspace-level permissions add a further layer, controlling access to notebooks, jobs, and other workspace objects independently of Unity Catalog's data and model permissions. Because these are separate permission systems, both layers must be configured consistently, or gaps can appear between what a workspace principal is authorized to run and what Unity Catalog permits it to access.
Where AI agents create a governance gap
AI agents built on Databricks typically call Unity Catalog-registered functions as tools. This tool access is governed through the same permission model applied to other data objects, rather than through a separate agent-specific permission system. In practice, an agent inherits the permissions of the caller or service principal under which it executes, so governance of agent behavior depends heavily on how that identity is scoped.
Unity Catalog can determine which functions, tables, or models an agent is permitted to reach, but it does not govern the process behavior of a running cluster or serverless session once compute has been provisioned. Model Serving endpoints and the Mosaic AI Gateway add request-level controls, such as rate limiting and safety filtering, for deployed models. These operate at the inference-request layer and are distinct from, not a substitute for, Unity Catalog's data-access permissions.
Lineage captured by Unity Catalog also has a boundary: it reflects data movement and query relationships, not an agent's internal decision logic or the sequence of intermediate tool calls it made to reach a result.
Governance gap: Unity Catalog and runtime controls jointly determine what an agent may access and how compute is configured, but neither layer, by itself, records or constrains how an agent uses that access during execution.
Evaluating native controls against governance requirements
Unity Catalog combined with cluster policies and workspace permissions provides substantial native governance for many Databricks AI workloads. Coverage includes data access, model versioning permissions, and audit trails through system tables that expose access and billing information for compliance reporting.
For organizations whose compliance requirements stop at controlling what data and tools an agent can reach, and at maintaining an auditable record of access, these native mechanisms may be sufficient.
The gap becomes material when requirements extend to:
- Restricting or approving specific tool calls at execution time
- Monitoring an agent's behavior across a session rather than only at the point of access
- Enforcing least-privilege identity scoping that adapts as an agent moves between tasks
These are runtime governance functions rather than data-access governance functions, and they sit outside what Unity Catalog and cluster policies are designed to provide. Trussed AI focuses on this runtime layer, applying policy enforcement, tool approval workflows, and audit logging to agent behavior during execution, as a complement to Unity Catalog's data-layer controls rather than a replacement for them.
Implementation considerations for platform teams
- Define and enforce cluster policies organization-wide before deploying AI workloads, since ad hoc cluster configurations can bypass intended governance.
- Review service principal and identity scoping for AI agents separately from human user permissions to avoid overly broad tool access.
- Validate compute access mode, shared or single-user, per workload type, since it determines whether Unity Catalog's fine-grained access control is active.
- Correlate Unity Catalog audit and lineage data with compute-layer logs, including cluster events and job runs, to reconstruct full AI workload behavior.
- Reconcile workspace permission structures with Unity Catalog permission structures, as gaps between the two are a common source of unintended access.
Frequently asked questions
Does Unity Catalog govern how an AI agent behaves at runtime?
No. Unity Catalog governs which data, models, and functions an agent is permitted to access, based on the identity it executes under. It does not govern the agent's process behavior, decision sequence, or tool-call pattern once compute has been provisioned for that session.
Is the Mosaic AI Gateway a substitute for runtime governance?
No. The AI Gateway applies request-level controls such as rate limiting and safety filtering to model serving endpoints. These controls operate at the inference-request layer and are separate from both Unity Catalog's data-access governance and broader runtime execution controls.
How should service principal permissions be scoped for AI agents?
Service principal or identity credentials used by an AI agent should be reviewed independently of human user permissions, since an agent inherits whatever access that identity has been granted. Overly broad scoping is a common source of unintended data or tool access.
Governing AI agents beyond data access
Unity Catalog and Databricks runtime controls establish what an agent can access. Closing the gap on how an agent behaves during execution requires dedicated runtime governance.
Explore Runtime Governance