Mirrored Metadata Catalog for Databricks - No Data Agent?

Hi dbeavon3​,

I think there is an important distinction here between a Mirrored Azure Databricks Catalog and a Fabric Mirrored Database.

Microsoft’s current Fabric Data Agent data-source documentation lists Lakehouse, Warehouse, Fabric SQL Database and Mirrored Database as supported SQL sources.

For a Lakehouse specifically, Data Agent queries the Delta tables that are exposed through the Lakehouse SQL analytics endpoint.

Azure Databricks catalog mirroring works a little differently. Microsoft’s mirroring documentation explains that Databricks catalog mirroring synchronizes the Unity Catalog metadata while the underlying data is accessed through OneLake shortcuts.

I do not currently see Mirrored Azure Databricks Catalog listed as a Data Agent source that can be attached directly.

However, Microsoft does document a supported bridge: you can create Lakehouse shortcuts to the mirrored Databricks catalog.

The important part is where those shortcuts are created.

If the Databricks Delta tables are added under the Lakehouse Tables section, Fabric can register them as tables and expose them through the Lakehouse SQL analytics endpoint.

I would test this first:

1. Open the Lakehouse SQL analytics endpoint.
2. Check whether one of the Databricks-backed shortcut tables appears there.
3. Run a simple SELECT against it.
4. If that works, try adding that Lakehouse - rather than the mirrored Databricks catalog item itself - as the Data Agent source.

Microsoft’s Lakehouse shortcut documentation notes that shortcuts in the Tables section can be queried through both Spark and the SQL analytics endpoint, while shortcuts under Files are not automatically registered as SQL tables.

So my suspicion would be:

Databricks Unity Catalog
-> mirrored Databricks catalog
-> Lakehouse table shortcut
-> Lakehouse SQL analytics endpoint
-> Fabric Data Agent

rather than attaching the mirrored catalog directly to the Data Agent.

If the shortcut table is already visible and queryable from the SQL analytics endpoint but the Data Agent still gives “Couldn't add data source,” then I think you have a much stronger case for a current Data Agent Preview limitation or product issue.

AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.

Hi dbeavon3​,

Are you using a mirrored databricks catalog, or are you using ADLS shortcuts in a lakehouse? These are both different things. 
Mirroring would bring the data natively into OneLake, where as shortcuts leave the data outside of OneLake. 

Hi dbeavon3​ ,

If it helps, here's a quick notebook snippet to check which Delta features your Databricks tables are using. Attach the lakehouse with the shortcuts and run it.

import pandas as pd

rows = []
for db in spark.catalog.listDatabases():
    for t in spark.catalog.listTables(db.name):
        name = f"`{db.name}`.`{t.name}`"
        try:
            d = spark.sql(f"DESCRIBE DETAIL {name}").collect()[0].asDict()
            props = d.get("properties") or {}
            rows.append({
                "table": name,
                "reader_version": d.get("minReaderVersion"),
                "writer_version": d.get("minWriterVersion"),
                "features": ", ".join(d.get("tableFeatures") or []),
                "column_mapping": props.get("delta.columnMapping.mode", ""),
                "deletion_vectors": props.get("delta.enableDeletionVectors", ""),
                "error": ""
            })
        except Exception as e:
            rows.append({"table": name, "error": str(e)[:300]})

display(pd.DataFrame(rows))

Then run this on the SQL analytics endpoint to see which tables actually made it through.

SELECT s.name AS schema_name, t.name AS table_name

FROM sys.tables t

JOIN sys.schemas s ON t.schema_id = s.schema_id

ORDER BY schema_name, table_name;

Any table that shows up in the notebook but not on the SQL endpoint is a likely suspect, and the features column should hint at why. Tables with a high reader version or features like deletion vectors or column mapping are the first ones I'd look at.

AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.