Forum Discussion
Helper IIFabric - MirroredAzureDatabricksCatalog DataSource/Catalog name
Hi,
I could not find any api for artifact type Azure Databricks mirroring in Microsoft Fabric, that ouputs information about Datasource metadata. We have successfully created Azure Databricks (Unity Catalog) mirrored items in Fabric. These items are visible via:
/v1/workspaces/{workspaceId}/items
and appear with the item type:
MirroredAzureDatabricksCatalog
However, there is no REST API to retrieve:
- Datasource / connection metadata (Databricks workspace, catalog, auth type)
Additionally, these items do not appear in:
/v1/workspaces/{workspaceId}/mirroredDatabases
which only returns items of type MirroredDatabase.
Is there any api that can provide catalog name or workspace URL(Azure Databricks Workspace URL)? Why I need it because I am building Lineage of Fabric Artifacts for Governance perspective so if you know there is an alternative please let me know!
TIA!
Hi Max_701 , Thank you for reaching out to the Microsoft Community Forum.
This is expected behavior. Currently, Microsoft Fabric does not expose source or connection metadata for Azure Databricks (Unity Catalog) mirrored items through any public REST API. Although these artifacts appear in /v1/workspaces/{workspaceId}/items as MirroredAzureDatabricksCatalog, they are modeled as a different artifact type than database mirrors and therefore do not appear in /mirroredDatabases. The getDefinition endpoint also does not return any definition parts for this item type, even after the long running operation completes successfully.
For lineage or governance scenarios, this means there is currently no supported API to retrieve Databricks workspace URL, catalog name or authentication details for these mirrored items. The only partial alternative is the Admin Workspace Info / Scanner API with lineage=true, which may surface high level upstream relationships but not detailed Databricks metadata.
If that is insufficient, the best thing to do for this to happen is submitting this clearly via Microsoft Fabric Ideas if you haven’t already, and get as many upvotes as possible. Microsoft monitors it for feature updates and enhancements, who knows, with enough upvotes this idea may be implemented as feature in the near future.
6 Replies
- burakkaragoz
Super User
Hi Max_701 ,
You have correctly identified a gap in the current API surface. Since MirroredAzureDatabricksCatalog is a distinct (and relatively new) item type, it does not currently populate the generic /mirroredDatabases endpoint, which is primarily designed for the initial set of mirrored engines (like SQL Database or Cosmos DB).
To get the metadata you need (Catalog Name, Source URL) for your Lineage solution, you likely cannot rely on the "Get Item" summary endpoints. You need to access the item's internal definition.
Here is the alternative approach that typically works for retrieving "hidden" configuration details in Fabric:
1. Use the "Get Item Definition" Endpoint Instead of just getting the item metadata, request its definition.
Method: POST
Endpoint: /v1/workspaces/{workspaceId}/items/{itemId}/getDefinition
2. Parse the Response This API returns a JSON object containing parts. These parts represent the internal files that make up the item.
Look for a part named model.json, context.json, or definition.json.
You may need to Base64 decode the content of these parts.
Inside the decoded JSON, you should find the connectionId and typically the source configuration (which contains the Databricks Catalog Name and Workspace URL).
3. Correlate with Connections API If the definition only gives you a connectionId, you will then need to call the Connections API to resolve that ID into a readable endpoint URL:
Endpoint: /v1/connections/{connectionId}
For Governance/Lineage specifically: If you are building an automated scanner, also check the Admin Scanner API (POST /v1/admin/workspaces/getInfo?lineage=true). While it might not fully support deep Databricks mirroring metadata yet, it is the intended path for extracting "Upstream Data Source" information without parsing individual item definitions.
If the Admin API returns empty upstreamSources for this item type, the getDefinition method is your best immediate workaround.
If my response resolved your query, kindly mark it as the Accepted Solution to assist others. Additionally, I would be grateful for a 'Kudos' if you found my response helpful.
This response was assisted by AI for translation and formatting purposes.
Max_701Hi burakkaragoz - Thanks for your response! I was testing this approach but there seems to be another issue when I use this endpoint - /v1/workspaces/{workspaceId}/items/{itemId}/getDefinition - it works well for most of the artifacts like lakehosue or warehouse but when I pass item id of a Azure Databricks Mirrored Item - it provides null as response (status code is 202)
- burakkaragoz
The 202 Accepted status code is the clue here! It is not a failure, but it is not an immediate success either.
In the Fabric REST API pattern, a 202 response means the request triggered a Long Running Operation (LRO). The server is essentially saying: "I have started fetching the definition, but I am not done yet." Unlike standard items (Lakehouse/Warehouse) which return the definition instantly (200 OK), mirrored items often require backend processing that forces this asynchronous flow.
The Missing Step: The Polling Pattern When you get a 202 response with a null body, you must look at the Response Headers to get the actual data.
Find the Location Header: Inspect the headers of that 202 response. You should see a header named Location (or Operation-Location) containing a URL.
Poll that URL: Make a GET request to that URL.
Check Status:
If it returns 202 again: Wait a few seconds and try again.
If it returns 200 OK: The Body of this response will contain the Item Definition (base64 parts) you are looking for.
Plan B: The Admin Scanner API If you follow the polling process and the final payload is still empty (which can happen with Preview artifacts like MirroredAzureDatabricksCatalog), then the Definition API is likely not fully supported for this item type yet.
In that case, your fallback for Lineage/Governance is the Admin Workspace Info API, which is designed to expose upstreamSources without needing the item definition:
Method: POST
Endpoint: /v1/admin/workspaces/getInfo?lineage=true
Body: { "workspaces": ["<your-workspace-id>"] }
This endpoint bypasses the specific item definition logic and queries the platform's lineage graph directly.
Try the polling method on the Location header first, that is the standard way to handle a 202!
If my response resolved your query, kindly mark it as the Accepted Solution to assist others. Additionally, I would be grateful for a 'Kudos' if you found my response helpful.
This response was assisted by AI for translation and formatting purposes.
v-hashadapuCommunity Support
Hi Max_701 , Thank you for reaching out to the Microsoft Community Forum.
This is expected behavior. Currently, Microsoft Fabric does not expose source or connection metadata for Azure Databricks (Unity Catalog) mirrored items through any public REST API. Although these artifacts appear in /v1/workspaces/{workspaceId}/items as MirroredAzureDatabricksCatalog, they are modeled as a different artifact type than database mirrors and therefore do not appear in /mirroredDatabases. The getDefinition endpoint also does not return any definition parts for this item type, even after the long running operation completes successfully.
For lineage or governance scenarios, this means there is currently no supported API to retrieve Databricks workspace URL, catalog name or authentication details for these mirrored items. The only partial alternative is the Admin Workspace Info / Scanner API with lineage=true, which may surface high level upstream relationships but not detailed Databricks metadata.
If that is insufficient, the best thing to do for this to happen is submitting this clearly via Microsoft Fabric Ideas if you haven’t already, and get as many upvotes as possible. Microsoft monitors it for feature updates and enhancements, who knows, with enough upvotes this idea may be implemented as feature in the near future.
- Max_701
Hi v-hashadapu - Thanks for confirming that this isn't possible(atleast for time being) 🙂
