Organizations increasingly use Microsoft Fabric and Azure Databricks together so teams can choose the tools that best fit their data engineering, analytics, and AI workloads while working from a common data foundation.
OneLake external-location support for Azure Databricks Unity Catalog is now generally available in Microsoft Fabric, with OneLake Security enabled by default for Azure Databricks Storage items. On the Azure Databricks side, the feature is in preview.
With this integration, supported Unity Catalog-managed data can be stored directly in OneLake. To understand the setup and supported scenarios, Connect Azure Databricks to a OneLake external location.
Why this matters
Data teams often need Azure Databricks and Microsoft Fabric in the same architecture. Without a shared storage approach, organizations may create copies of the same data for different platforms, introduce pipelines whose primary purpose is synchronization, and manage access independently across multiple storage locations.
Using OneLake as managed storage for supported Unity Catalog objects gives organizations a common physical data foundation while preserving flexibility in the services used above it. Azure Databricks continues to provide compute and Unity Catalog management, while OneLake provides the storage destination and Fabric-side data-access controls.
What's new in this release
This release builds on the Beta announcement, which introduced OneLake external-location support for Unity Catalog.
OneLake Security is now enabled by default for Azure Databricks Storage items, so no user gets access when the item is created. Fabric workspace roles, including Admin, Member, and Contributor, cannot grant access to the data stored in Azure Databricks Storage item and managed through Unity Catalog. Only the Azure Databricks identity can read and write to Azure Databricks Storage items.
The updated security experience helps platform teams move from testing connectivity to planning controlled adoption.
One shared data foundation, with flexibility above it
Use OneLake as the storage foundation
Azure Databricks workloads can create supported Unity Catalog-managed data directly in OneLake. A Unity Catalog external location points to an Azure Databricks Storage item in OneLake, and a catalog or schema can use that external location as managed storage.
Continue using Unity Catalog-managed objects
Teams can continue using Azure Databricks and Unity Catalog to create and manage supported managed tables, volumes, views, materialized views, and streaming tables. The objects remain managed through the Azure Databricks experience while their underlying supported data is stored in OneLake.
Keep platform responsibilities clear
Azure Databricks remains the environment for Databricks compute and Unity Catalog management. Microsoft Fabric provides the capacity-backed workspace and Azure Databricks Storage item, while OneLake provides the physical storage foundation.
How it works
- Provision storage in Microsoft Fabric: Create a capacity-backed Fabric workspace and an Azure Databricks Storage item. The storage item provides the OneLake location used by the integration.
- Configure the Azure Databricks identity: Use the managed identity of an Access Connector for Azure Databricks, or a service principal. Assign it the Viewer role on the Fabric workspace, then add it as the only member of a ReadWrite OneLake security role on the Azure Databricks Storage item.
- Create the Unity Catalog connection: In Azure Databricks, create a storage credential for the approved identity, and then create a OneLake external location that points to the Azure Databricks Storage item.
- Create Unity Catalog objects: Configure a catalog or schema to use the OneLake external location as managed storage. Data written by supported Azure Databricks workloads is then stored directly in OneLake.
Security
OneLake security is enabled by default for the Azure Databricks storage item, so no user gets access when the item is created. Fabric workspace roles (Administrator, Member, Contributor, and Viewer) cannot grant access to the data in Azure Databricks storage item. To keep your data secure and under Unity Catalog governance:
- Add only the Azure Databricks access connector's managed identity to the OneLake security role.
- Do not add any other users, groups or service principals.
- Fabric users should read data through mirrored items, not through the storage directly.
Only the Azure Databricks identity should access the storage item: Unity Catalog is the single place
where you grant, review, and revoke access to the data.
How to read the data in Fabric
Fabric users read the data through a mirrored Azure Databricks catalog item, not through the Azure
Databricks Storage item. The mirrored catalog item shows the Unity Catalog catalogs, schemas, and
tables you select in a Fabric workspace, without copying the data. Analysts and engineers can query it
through the SQL analytics endpoint, notebooks, and Power BI reports.
You can create the mirrored catalog item from either platform:
- From Azure Databricks, with Publish to Fabric: Run it on the catalog you want to share, and it
creates the mirrored catalog item in the Fabric workspace you choose. This route suits data
engineers who already work in Azure Databricks. - From Microsoft Fabric, as a new Mirrored Azure Databricks catalog item: Connect to your
Azure Databricks workspace, then choose the catalog, schemas, or tables to include. This route
suits Fabric administrators and analysts who manage the workspace.
Fabric readers use the mirrored catalog item. Never add them to the storage item's OneLake security
role, which is reserved for the Azure Databricks identity.
Get started
Ready to use OneLake as managed storage for Azure Databricks Unity Catalog objects?
Start with the Connect Azure Databricks to a OneLake external location tutorial and plan your deployment.
Microsoft Employee