Forum Discussion
About overwrite in pipeline
Hi, could anyone please clarify on why the data comes repeatedly while I choose overwrite in data pipeline? Overwrite shouldn't give duplicate isn't it?
mikeburek
Advocate III
Advocate IIIYou are using the "Copy data" activity and your destination is a table in a lakehouse?
I think the "overwrite" should not create duplicate data.
How do the numbers of rows compare from your source data to your destination table? Do you see an integer multiplier, like 2x or 3x as many records in your destination table, meaning all of your rows are being duplicated? Or do only some rows get duplicated?
Are you doing anything with schemas? Or leaving the destination table in the dbo schema?