Forum Discussion
Ingest data from Databricks (on Azure)
Hi royclack ,
Thanks for using Microsoft Fabric Community and sharing the details.
In addition to jennratten suggestions,
It’s important to ensure the proper permissions are in place for the Unity Catalog in Databricks. In addition to SELECT, the EXTERNAL USE SCHEMA permission must be explicitly granted for access to the schema in Unity Catalog. Without this, even if the catalog is mirrored, you might not be able to access the data.
As mentioned by smpa01 , it’s possible that mirroring is being blocked by your Databricks admin. If that’s the case, a Databricks Personal Access Token (PAT) paired with a service principal will be necessary to programmatically access the data. This allows you to bypass mirroring restrictions and directly consume the data in Fabric.Once the Databricks PAT is configured, make sure the Spark-JDBC driver is placed in the Lakehouse. This will allow you to connect to Databricks and retrieve data programmatically.
It’s also worth noting that Databricks mirroring only supports managed and external tables. If the table you're trying to access is temporary or not part of Unity Catalog, it won't be available for direct querying via Fabric.Some useful resources: Microsoft Fabric Mirrored Catalog From Azure Databricks (Preview) - Microsoft Fabric | Microsoft Learn
Troubleshoot Fabric Mirrored Databases - Microsoft Fabric | Microsoft Learn
After verifying the permissions, admin settings, and ensuring the proper table types are being mirrored, let us know if the issue persists.
Hope this helps. Please reach out for further assistance.
If this post helps, then please consider to Accept as the solution to help the other members find it more quickly and a kudos would be appreciated.Thank you.
Hi jennratten
Have been looking at the mirroring and changed tennent setting, to allow the feature outside of trial capacity usage. Working with my colleague we've setup a Databricks 14 day trial, added some data to create a table (managed type) under the default schema and mirrored the catalogue. It appears we are experiencing an issue, as we can see the replicated catalogue in Fabric but cannot view the data.
We have added permissions on the Databricks side to grant external schema and select permissions. In Fabric we get the message:
Couldn't access data. Check your permissions for this catalog or try again later.
Have you checked to confirm that users have the required USE CATALOG, USE SCHEMA, or SELECT permissions on the Databricks side?
The cluster might also not be configured to support fine-grained access control (FGAC) or identity passthrough - or there might be a network policy or firewall rule that is blocking access.
You can check your permission for databricks by querying with SQL.
SELECT * FROM information_schema.catalog_privileges
WHERE grantee = '<your_username>';
SELECT * FROM information_schema.schema_privileges
WHERE grantee = '<your_username>';
SELECT * FROM information_schema.table_privileges
WHERE grantee = '<your_username>';
You can also check via the databricks UI:
Navigate to the Unity Catalog section of the UI and reviewing the permissions for catalogs, schemas, and tables.