Forum Discussion
Incremental refresh causing duplicate rows
Hello, I've implemented incremental refreshes on one of our fact tables, called 'Transactions'. I can see on a given day there's been a change to some rows, and when this dataflow is consumed...
Duplication can only happen when partitions store overlapping date ranges. You must make sure that of the RangeStart and RangeEnd parameters one is inclusive and one is exclusive
For example
Date >=RangeStart and Date < RangeEnd
Date >RangeStart and Date <=RangeEnd
are both possible (but not at the same time!)
However
Date >=RangeStart and Date <=RangeEnd
will lead to duplicate rows and must be avoided.
This worked for me, thank you!