Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Data Days is here! Join us now for 60+ days of learning, challenges, and connection. Learn more

preshah

Extending interoperability: Azure Databricks can now store Unity Catalog managed tables directly in OneLake

As organizations scale their data and AI investments, they increasingly adopt a multi-platform approach, enabling teams to use the tools that best fit their needs. The challenge has been enabling both platforms to work from a single copy of data without duplicating storage or building complex pipelines.

 

With today's updates, Azure Databricks and Microsoft Fabric take a major step toward true bi-directional interoperability through OneLake. Reading data across platforms is already available: Onelake catalog federation (now generally available) lets Azure Databricks query OneLake data directly, and Mirrored Azure Databricks Catalog makes Unity Catalog tables stored in ADLS Gen2 accessible in Fabric. Now, we are taking this a step further — Azure Databricks can store Unity Catalog managed tables directly in OneLake:

 

OneLake external location support for Unity Catalog (Beta): Azure Databricks can now store Unity Catalog managed tables directly in OneLake. Expanding your Azure storage options alongside ADLS Gen2, your workloads can seamlessly target OneLake as an additional foundational storage layer for Unity Catalog.

 

Publish to Fabric (Preview): A streamlined workflow to create Mirrored Azure Databricks Catalog items directly from Azure Databricks Catalog Explorer. This works for Unity Catalog tables regardless of where they're stored, whether in OneLake or in ADLS Gen2. Once published, tables are immediately queryable across all Fabric workloads.

 

Why this matters

 

With support for both reading and storing directly in OneLake, Azure Databricks customers can now use OneLake as a native storage layer for their Delta tables without managing separate storage systems. This provides flexibility to store data in OneLake while using preferred tools in either Microsoft Fabric or Azure Databricks for each project. Because these remain fully managed Unity Catalog tables, all existing governance and optimization capabilities extend to data stored in OneLake.

 

Because there is a single copy of data, changes made in one environment are immediately reflected in the other. This eliminates the need for data movement, reduces duplication, and simplifies how organizations manage and govern their data estate.

 

And with Publish to Fabric, you don't have to choose between storage locations to get your tables into Fabric. Whether your Unity Catalog tables are backed by OneLake or ADLS Gen2, Publish to Fabric creates mirrored items in any Fabric workspace, initiated directly from the Azure Databricks experience. The result is the same data accessible in both Azure Databricks and Fabric with no copies, no movement, and no duplication.

 

This represents a shift toward a shared data foundation across platforms. Organizations no longer need to choose between tools or duplicate data to support different workloads.

 

Key capabilities

 

Store Unity Catalog managed tables directly in OneLake (Beta)

 

This is the core of the announcement. OneLake external location support enables you to create Unity Catalog managed tables stored in Microsoft OneLake. You create a UC external location mapped to a OneLake path, and from that point forward, all UC asset types (managed tables, views, materialized views, and streaming tables) are stored directly into OneLake.

Unlike the existing pattern where UC tables are stored in ADLS Gen2 and surfaced in Fabric through mirroring, this approach makes OneLake the primary storage destination. Data written by Azure Databricks lives directly in OneLake from the start.

 

Publish to Fabric: Surface OneLake backed tables in Fabric (Preview)

 

Once your Unity Catalog tables are stored in OneLake, you need a way to make them visible and queryable in Fabric. That's where Publish to Fabric comes in. While Mirrored Azure Databricks Catalog already enables reading UC tables in Fabric, Publish to Fabric provides a streamlined, Azure Databricks native workflow to create mirrored catalog items directly from Catalog Explorer.

 

Once published, Fabric polls Unity Catalog metadata to keep the mirrored catalog in sync. DDL changes made in Azure Databricks (new tables, dropped tables, and schema modifications) are reflected in Fabric at the next sync interval. You can also Refresh on the mirrored item to force an immediate sync. Tables are read-only in Fabric since the source of truth remains in Azure Databricks, and you can publish the same catalog to multiple Fabric workspaces independently.

 

Getting started

 

Part 1: Set up OneLake as UC storage

 

  • Configure identity and access — Create or identify a Managed Identity or Service Principal in Azure, and assign it a workspace role (Admin, Member, or Contributor) in your target Fabric workspace.
  • Provision storage in Fabric — Create an Azure Databricks Storage item in your Fabric workspace.
  • Create a storage credential and external location — In Azure Databricks, create a Unity Catalog storage credential and an external location pointing to OneLake.
  • Create catalogs and tables — Create Unity Catalog assets on the OneLake external location. Data written to these tables lands directly in OneLake with no copy or pipeline.

 

Part 2: Publish the catalog to Fabric

 

  • Grant permissions — Assign the required Unity Catalog permissions to your Service Principal or user identity so Fabric can access the catalog metadata.
  • Create a Fabric connection — In Fabric, set up a cloud connection to your Azure Databricks workspace.
  • Publish from Catalog Explorer — In Azure Databricks, select your catalog and use the Publish to Fabric workflow to push it to your chosen Fabric workspace.
  • Query in Fabric — Once published, tables are immediately queryable across all Fabric workloads including SQL analytics, notebooks, and Power BI.

For detailed step-by-step instructions, review Configuring OneLake external locations and Publish a Unity Catalog catalog to Microsoft Fabric.

 

Resources

 

We'd love to hear your feedback. How are you using Azure Databricks and Fabric together? What additional capabilities would make this integration more powerful for your team? Share your thoughts in the Fabric Ideas forum!