Forum Discussion

DuncanJ's avatar
DuncanJ
New Member
3 years ago
Solved

Copy data into lakehouse and warehouse - append or overwrite

When creating a data pipeline that copies data into a lakehouse, I can choose whether to Append or Overwrite data. If I choose to Overwrite, repeated runs of the pipeline do not create duplicate records.

 

However, there is no option to Append or Overwrite when creating a data pipeline that copies data into a warehouse. As a result, repeated runs of the pipeline create duplicate records in the warehouse.

 

 

Shouldn't the option to Append or Overwrite be present in the warehouse experience too?

 

Or have I misunderstood the intent? Any wisdom appreciated.

 

Thank you

 

Duncan

 

 

  • Duncan,

     

    I agree with you ) That's why it is in preview mode for now.

    I hope the Microsoft team sees your post and fixes this small but very important issue! !

    To be honest, I didn't notice it before.)