Could incremental refresh fail due to duplicate keys?

[AshuBrar]

Hi all,

I was in the middle of implementing incremental refresh when a note from the documentation caught my attention. I'm sharing the image below.

My question is: Will incremental refresh throw an error if the incremental rows have a key that already exists in the historical partition? Or will that error only occur in specific scenarios?

Thank you in advance.

[v-zhangti · Accepted solution]

Hi, @AshuBrar 

If the incremental rows have a key that already exists in the historical partition, then the incremental refresh will throw an error. However, this error will only occur if the "Allow incremental refresh to update rows" option is enabled in the dataset settings. If this option is disabled, then the incremental refresh will skip the rows that already exist in the historical partition and only add the new rows.

Best Regards,

Community Support Team _Charlotte

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.