Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.
When trying to access one lake Microsoft fabric (lakehouse) from databricks notebook to read data froom adls and write into fabric lakehouse getting this error: path has invalid authority error while reading and writing files to lakehouse
abfss://onelake.dfs.fabric.microsoft.com/path(actual path of file from lakehouse i am giving) has invalid authority.
I have followed steps from : https://learn.microsoft.com/en-us/fabric/onelake/onelake-azure-databricks
created cluster and enable Azure Data Lake Storage (ADLS) credential passthrough on your cluster in the Advanced Options, but i have databricks premium account for different microsoft account and fabric workspace on different microsoft account.
Is it because of that? do i need to create fabric and databricks account for same Microsoft ID, if i wanted to use different account then what is the way to do that? please help
Solved! Go to Solution.
Hi @KomalPawar
Thanks for using Microsoft Fabric Community.
Apologies for the inconvenience.
You're correct that having your Databricks workspace and Fabric workspace on different Azure Active Directory (AAD) tenants can cause the "invalid authority" error you're encountering.
While Microsoft Fabric offers seamless integration with Azure Databricks within the same tenant, there are currently limitations when working across different tenants. This means you cannot directly access an ADLS Gen2 storage account managed by a separate AAD tenant from your Databricks workspace, even with a premium account.
If you’re using different Microsoft accounts for Databricks and the fabric workspace, you might need to set up a service principal with the necessary permissions on the ADLS resource and configure Databricks to use this service principal for authentication.
For more details please refer to the following documents.
Link1 : Connect to Azure Data Lake Storage Gen2 and Blob Storage.
I hope this information helps. Please do let us know if you have any further questions.
Thanks.
Hi @KomalPawar
Thanks for using Microsoft Fabric Community.
Apologies for the inconvenience.
You're correct that having your Databricks workspace and Fabric workspace on different Azure Active Directory (AAD) tenants can cause the "invalid authority" error you're encountering.
While Microsoft Fabric offers seamless integration with Azure Databricks within the same tenant, there are currently limitations when working across different tenants. This means you cannot directly access an ADLS Gen2 storage account managed by a separate AAD tenant from your Databricks workspace, even with a premium account.
If you’re using different Microsoft accounts for Databricks and the fabric workspace, you might need to set up a service principal with the necessary permissions on the ADLS resource and configure Databricks to use this service principal for authentication.
For more details please refer to the following documents.
Link1 : Connect to Azure Data Lake Storage Gen2 and Blob Storage.
I hope this information helps. Please do let us know if you have any further questions.
Thanks.
Hi @KomalPawar
Glad that your query got resolved.
Please continue using Fabric Community for any help regarding your queries.
Thanks.