Incremental Refresh Overlapping/Duplication of Rows

Hi Arun_Benny_TS,

Thank you for reaching out to the Microsoft fabric community forum. Thank you mariussve1, for your inputs on this issue.

After thoroughly reviewing the details you provided, I have identified few workarounds that may help resolve the issue. Please follow these steps:

The Incremental Refresh Overlapping/Duplication of Rows error in Power BI usually occurs due to improper partitioning or misaligned filter logic during the implementation of Incremental Refresh.

This problem often arises when the RangeStart and RangeEnd parameters are not correctly aligned with the data source's partitions, leading to overlapping data segments or duplicated rows. Ensuring precise filter alignment and verifying partition boundaries can help mitigate this issue.

Ensure your query correctly applies the RangeStart and RangeEnd parameters in a date-based filter. Check Applied Steps in Power Query by navigating to Transform Data → Check Filtered Rows.

Ensure that the date filter applied in Power Query (RangeStart and RangeEnd parameters) correctly partitions the data. Typically, it should be: FilteredRows = Table.SelectRows(Source, each [Date] >= RangeStart and [Date] < RangeEnd). The less than (<) RangeEnd is crucial to avoid overlapping.

If using a native SQL query, ensure the filter is correctly applied in the WHERE clause: “WHERE DateColumn >= @RangeStart AND DateColumn < @RangeEnd.” Avoid <= RangeEnd, as it may cause overlapping rows.

Ensure the RangeStart and RangeEnd parameters are correctly set. The RangeStart should filter dates AFTER the specified date, and the RangeEnd should filter dates BEFORE OR EQUAL TO the specified date. This prevents overlapping intervals that can cause duplication.

The document below indicates that the refresh fails due to duplicate values,

Kindly review the documentation link provided for a better understanding:

Troubleshoot incremental refresh and real-time data - Power BI | Microsoft Learn

I hope this could resolve your issue, if you need any further assistance, feel free to reach out.
If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.

Thank you for using Microsoft Community Forum.