Forum Discussion

ToddChitt's avatar
ToddChitt
Super User
1 year ago
Solved

Duplicate rows via a staging table

This solution involves quite a few different technologies and services from Micorsoft, but I thought this was the most appropriate forum for the question. I have an on-premise source system that is...
data engineering
Lakehouse
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi ToddChitt,

     

    Thank you for reaching out to Microsoft Fabric Community Forum.

     

    Verify whether the FULL and INCREMENTAL loads are overlapping or interfering with each other. If the FULL load takes longer than expected, an INCREMENTAL load might begin while the table is still being truncated and reloaded, potentially causing duplicates.

    If there is no overlap and the issue persists, I recommend raising a Microsoft Fabric support ticket for further investigation.

    How to create a Fabric and Power BI Support ticket - Power BI | Microsoft Learn

     

    Regards,

    Vinay Pabbu

ToddChitt's avatar
ToddChitt
Super User
1 year ago

I can say without any doubt that there is NO overlap between the FULL and DELTA loads. The former happens ONLY on Sundays mornings, t)akes about 90 minutes) and the DELTA happens Monday through Saturday evenings (and takes about 20 to 30 minutes).