Forum Discussion

Srisakthi's avatar
Srisakthi
Super User
1 year ago
Solved

Mirrored Azure Databricks Catalog

Hi Everyone,

 

Has anyone tried using the latest feature of Mirroring Azure databricks unity catalog ?. I was able to successfully mirror unity catalog tables, but I'm facing an error with Unmanaged tables. I have tried User identity as well Service Principal. Enabled “External Data Access” on the metastore, “EXTERNAL_USE_SCHEMA” , “SELECT”,“USE” permissions has been provided already.

I can query the managed table but i cant query unmanaged table. ADLS(unmanaged table) is not under any private network .

Please suggest if i have missed any permissions.

 

Thanks,

Srisakthi

data engineering
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi Srisakthi 

     

    I haven't used this preview feature yet. I went through the relevant documentations, it seems you’ve already covered many of the necessary steps for setting up and querying unmanaged (external) tables in Azure Databricks Unity Catalog. Here are a few additional things you might want to check:

    1. Storage Credential and External Location: Ensure that the storage credential and external location are correctly configured and associated with the external table. The external location should be accessible with the storage credential you have set up.

    2. Permissions on External Location: Verify that the necessary permissions are granted on the external location. This includes ensuring that the user or service principal has theREAD and WRITE permissions on the Azure Data Lake Storage (ADLS) path. 

    3. Schema Permissions: Double-check that the EXTERNAL_USE_SCHEMA, SELECT, and USE permissions are correctly applied to the schema containing the external table.

    4. Network Configuration: Since your ADLS is not under any private network, ensure that there are no network restrictions or firewall rules that might be blocking access to the external storage.

    5. Error Messages: Review the error messages when you failed to query from the external tables. It might give more insight into what is going wrong. Did you see any specific error messages? 

       

    If you’ve verified all these aspects and are still facing issues, it might be helpful to reach out to Fabric support for more detailed troubleshooting. 

     

    Best Regards,
    Jing
    If this post helps, please Accept it as Solution to help other members find it. Appreciate your Kudos!

13 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable
    1 year ago

    Hi Srisakthi 

     

    I haven't used this preview feature yet. I went through the relevant documentations, it seems you’ve already covered many of the necessary steps for setting up and querying unmanaged (external) tables in Azure Databricks Unity Catalog. Here are a few additional things you might want to check:

    1. Storage Credential and External Location: Ensure that the storage credential and external location are correctly configured and associated with the external table. The external location should be accessible with the storage credential you have set up.

    2. Permissions on External Location: Verify that the necessary permissions are granted on the external location. This includes ensuring that the user or service principal has theREAD and WRITE permissions on the Azure Data Lake Storage (ADLS) path. 

    3. Schema Permissions: Double-check that the EXTERNAL_USE_SCHEMA, SELECT, and USE permissions are correctly applied to the schema containing the external table.

    4. Network Configuration: Since your ADLS is not under any private network, ensure that there are no network restrictions or firewall rules that might be blocking access to the external storage.

    5. Error Messages: Review the error messages when you failed to query from the external tables. It might give more insight into what is going wrong. Did you see any specific error messages? 

       

    If you’ve verified all these aspects and are still facing issues, it might be helpful to reach out to Fabric support for more detailed troubleshooting. 

     

    Best Regards,
    Jing
    If this post helps, please Accept it as Solution to help other members find it. Appreciate your Kudos!

    • Srisakthi's avatar
      Srisakthi
      Super User
      1 year ago

      Hi Jing, Yeah as I mentioned earlier all those options are enabled already. Here is the screenshot of the error.

       

       

      Thanks,

      Srisakthi

      • Anonymous's avatar
        Anonymous
        Not applicable
        1 year ago

        Hi Srisakthi 

        Have you tried using other tools to connect to its SQL analytics endpoint and query data from the unmanaged tables? Would it succeed? 

         

        Best Regards,
        Jing

  • aaron2's avatar
    aaron2
    Regular Visitor
    1 year ago

    Experiencing the same issue. Complete the catalog mirror connection, but receive same error message as the others and also don't see any tables in the sql endpoint. When clicking 'Go to mirrored database' in the SQL endpoing page, nothing ever loads.

  • Mahens's avatar
    Mahens
    Microsoft Employee
    1 year ago

    For all the people facing issue in previewing data. Please follow instructions

     

    1. You must enable external data access on the metastore. For more information, see  

    Enable external data access on the metastore 

    2. You must have the EXTERNAL USE SCHEMA privilege on the schema in Unity Catalog that contains the tables that will be accessed from Fabric. For more information, see https://learn.microsoft.com/en-us/azure/databricks/external-access/admin#external-schema

     

    Currently we dont support for storage account behind firewall. Please ensure your storage account is enabled with public network access.