Forum Discussion

jaryszek's avatar
jaryszek
Super User
8 months ago
Solved

Why DataLake over OneLake after adding new schema is not appearing in semantic model?

Hi Guys,

I am using DataLake over OneLake. When a new schema is built as delta lake files:


I am doing :

1. Refresh the DataLake house within Fabric

2. Going to the semantic model built on top of this data lake and clicking refresh. 


Edit tables -> refresh

 

But still new schema is not there. 
After few trials it appears. 

Is there any way to see this immidietely after refresh? 
Did anybody has a situation like that? 

Best,
Jacek

 

  • I agree with anilelmastasi  here. as Fabric uses an asynchronous metadata refresh mechanism for Lakehouse objects. This means:

    • When new files or schemas are added to the Lakehouse, Fabric updates its internal metadata catalogs in the background, not instantly.
    • The Lakehouse UI may show updated files immediately, but the semantic model relies on metadata services that may still be catching up.
    • When you click 'Refresh' in the semantic model, it pulls from that internal metadata catalog—not directly from the lake folder structure.

    As a result, there can be a delay before the semantic model becomes aware of new schemas, even though the Lakehouse has already refreshed. some generic suggested actions are:

    • Closing and reopening the semantic model
    • Re-entering the “Edit tables” view
    • Refreshing the entire Lakehouse again
    • Waiting a few minutes

    These actions don’t force a metadata refresh, but they may coincide with the background refresh completing.

     

     

     

    Thanks,

    Prashanth

6 Replies

  • v-prasare's avatar
    v-prasare
    Community Support

    Hi jaryszek,

    We would like to confirm if our community members answer resolves your query or if you need further help. If you still have any questions or need more support, please feel free to let us know. We are happy to help you.

     

    anilelmastasi, thanks for your prompt response

     

     

    Thank you for your patience and look forward to hearing from you.
    Best Regards,
    Prashanth Are
    MS Fabric community support

    • jaryszek's avatar
      jaryszek
      Super User

      Hello,

      not it is not working for me. What Microsoft advices?

      Best,
      Jacek

      • v-prasare's avatar
        v-prasare
        Community Support

        I agree with anilelmastasi  here. as Fabric uses an asynchronous metadata refresh mechanism for Lakehouse objects. This means:

        • When new files or schemas are added to the Lakehouse, Fabric updates its internal metadata catalogs in the background, not instantly.
        • The Lakehouse UI may show updated files immediately, but the semantic model relies on metadata services that may still be catching up.
        • When you click 'Refresh' in the semantic model, it pulls from that internal metadata catalog—not directly from the lake folder structure.

        As a result, there can be a delay before the semantic model becomes aware of new schemas, even though the Lakehouse has already refreshed. some generic suggested actions are:

        • Closing and reopening the semantic model
        • Re-entering the “Edit tables” view
        • Refreshing the entire Lakehouse again
        • Waiting a few minutes

        These actions don’t force a metadata refresh, but they may coincide with the background refresh completing.

         

         

         

        Thanks,

        Prashanth

  • Hello jaryszek ,

     

    This has detailed background and metadata cause this problem but when we look what is going on the background, there are some process'.

     

    Lakehouse metadata refresh is async, when you click lakehouse refresh, Fabric scans the system-updates internal metadata caches- and does not guarantee immediate schema propagation. 

     

    When you click semantic model-edit tables-refresh, the model reads cached tabled definitions and do not force a re-scan of OneLake.

     

    So semantic model refresh is not equal to refreshing metadata. There can be delay.

     

    Can you try closing and reopening semantic model tab? After that you can click edit tables.

  • v-prasare's avatar
    v-prasare
    Community Support

    Hi jaryszek,

    We would like to confirm if our community members answer resolves your query or if you need further help. If you still have any questions or need more support, please feel free to let us know. We are happy to help you.

     

     

     

    Thank you for your patience and look forward to hearing from you.
    Best Regards,
    Prashanth Are
    MS Fabric community support