Forum Discussion

FabianSchut's avatar
FabianSchut
Solution Sage
1 year ago
Solved

Mirrored Azure Databricks catalog is missing columns in Fabric

Hi all,   We have a Mirrored Azure Databricks catalog running in Fabric. Today we noticed that there are several columns missing. When we look closer at the columns that are missing, the column t...
data engineering
  • nilendraFabric's avatar
    nilendraFabric
    1 year ago

    Hello FabianSchut 

     

    Seems like `TIMESTAMP_NTZ` is not fully compatible with Fabric’s SQL analytics endpoints.This incompatibility can result in columns being excluded or not displayed properly during schema synchronization.

     


    is this possible for you to convert TIMESTAMP_NTZ` column to a `TIMESTAMP` in Databricks.

     

    When you do this conversion in Databricks 

    it will interpret the timestamp using the current session time zone, because `TIMESTAMP_NTZ` values do not include time zone information and must be mapped to a particular zone when converted.

     

    Thanks 

v-karpurapud's avatar
v-karpurapud
Community Support
1 year ago

Hi FabianSchut 


Thanks for reaching out to Microsoft Fabric Community Forum.

 

Thank you for your response nilendraFabric ! I appreciate your insights.

 

Additionally, if you do not prefer to convert the data type, consider the following steps:

 

  1. Check Permissions:
    Ensure that the necessary permissions are granted for the schema in Unity Catalog that contains the tables to be accessed from Fabric. The EXTERNAL USE SCHEMA privilege must be explicitly granted.

  2. Configure Replication:
    Verify that the tenant setting "Mirrored Azure Databricks Catalog" is turned on in the Microsoft Fabric Admin Portal.

 

Furthermore, we would like to provide some reference links that may be useful:

Microsoft Fabric Mirrored Databases From Azure Databricks (Preview) Tutorial - Microsoft Fabric | Microsoft Learn
Microsoft Fabric Mirrored Catalog From Azure Databricks (Preview) - Microsoft Fabric | Microsoft Learn


If my response has resolved your query, please mark it as the Accepted Solution to help others. Additionally, I would appreciate a 'Kudos' if you found my response helpful.

 

Thank you!