Forum Discussion

jaryszek's avatar
jaryszek
Super User
5 months ago
Solved

DirectLAke over OneLake how to use incremental load?

Hello,   How to use parameters/incremental load in OneLake ?   Best, Jacek
Need Help
  • v-dineshya's avatar
    v-dineshya
    5 months ago

    Hi jaryszek ,

    Thank you for reaching out to the Microsoft Community Forum.

     

    Hi anilelmastasi , Thank you for your prompt response. 

     

    Hi jaryszek ,


    What Is the Correct Refresh Order?

    1. Raw Bronze --> append new files only
    2. Silver --> upsert changes (MERGE)
    3. Gold --> aggregations, denormalization

     

    How Consistency Is Maintained in Delta?

    Delta Lake ensures:

    1. ACID Transactions: Every write creates, _delta_log/NNNN.json, snapshot, commit metadata and optional checkpoint.

    2. Snapshot Isolation: Readers always see the latest completed version.

    3. No Locking Problems: Writers do not block readers.

     

    I hope this information helps. Please do let us know if you have any further queries.

     

    Regards,

    Dinesh

     

     

v-dineshya's avatar
v-dineshya
Community Support
5 months ago

Hi jaryszek ,

Thank you for reaching out to the Microsoft Community Forum.

 

Hi anilelmastasi , Thank you for your prompt response. 

 

Hi jaryszek ,


What Is the Correct Refresh Order?

1. Raw Bronze --> append new files only
2. Silver --> upsert changes (MERGE)
3. Gold --> aggregations, denormalization

 

How Consistency Is Maintained in Delta?

Delta Lake ensures:

1. ACID Transactions: Every write creates, _delta_log/NNNN.json, snapshot, commit metadata and optional checkpoint.

2. Snapshot Isolation: Readers always see the latest completed version.

3. No Locking Problems: Writers do not block readers.

 

I hope this information helps. Please do let us know if you have any further queries.

 

Regards,

Dinesh

 

 

v-dineshya's avatar
v-dineshya
Community Support
5 months ago

Hi jaryszek ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.

 

Regards,

Dinesh