Forum Discussion

tolgakurt's avatar
tolgakurt
Frequent Visitor
1 year ago
Solved

DirectLake Auto Refresh Error

Hello.   I created a lakehouse on Fabric workspace and took the datasets here to the semantic model and visualized them via PowerBI. I added a new column to the delta format transaction table in La...
capacities
one lake
  • v-prasare's avatar
    v-prasare
    1 year ago

    tolgakurt updating the Semantic Model is the key step after making schema changes in your Fabric Lakehouse. 

     

    always Update the Schema in the Semantic Model After Schema Changes, Any time you rename, add, or remove columns in your Lakehouse table, make sure to refresh and update the dataset schema in Power BI.

     

     

     

    Thanks,

    Prashanth

    MS Fabric community support

     

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query

     

     

burakkaragoz's avatar
burakkaragoz
Super User
1 year ago

Hi

 

I see you're experiencing a DirectLake Auto Refresh error after modifying a delta table schema in your Fabric Lakehouse. Let me explain what's happening and how to fix it.

The Issue

Based on your description, you:

  1. Created a Lakehouse in your Fabric workspace
  2. Connected the datasets to a semantic model for PowerBI visualization
  3. Made schema changes in your delta table using Spark (added a new column and renamed "isFirstTrx" to "FirstTrx")
  4. After these changes, DirectLake Auto Refresh started failing with an error about not being able to access the source column

    Root Cause

    The error occurs because DirectLake has cached the previous schema of your Transaction table. When you rename or modify columns, the auto sync mechanism can't find the original column names it's expecting ("isFirstTrx"), causing the refresh to fail.

    The error message indicates this clearly: "We cannot access the source column 'isFirstTrx' of delta table 'Transaction' referenced by table 'Transaction'."

    Solution

    Here's how to fix this issue:

    1. Refresh the semantic model schema:
      • Go to your semantic model in Power BI
      • Choose "Refresh structure" (not just regular refresh)
      • This forces the semantic model to re-read the table schemas
      • Update column references in the semantic model:
        • Open the semantic model in editing mode
        • Find any measures, relationships, or calculated columns that reference the old column name
        • Update all references from "isFirstTrx" to "FirstTrx"
        • If the above doesn't work, try this more thorough approach:
          • Go to your semantic model settings
          • Temporarily disable DirectLake connection
          • Remove the problematic table from your model
          • Save changes
          • Re-add the table with its new schema
          • Re-enable DirectLake
          • Rebuild any necessary relationships and measures
          • Check for hidden dependencies:
            • Look for any calculated tables or measures that might be referencing the old column
            • Inspect relationships that might be using the renamed column

              In some cases, you may need to clear the DirectLake cache to force a fresh schema read. You can do this by running the following command in a Spark notebook connected to your Lakehouse:

               

               
              python
              spark.sql("CALL lakehouse.system.sync()")
               

              This forces Fabric to synchronize the metadata and should resolve schema discrepancy issues after column modifications.