Forum Discussion
alynch
Advocate I
Advocate ICopy data activity not overwriting
We have pipelines that have been running for months without issue. Overnight some copy data activities from lakehouse to lakehouse are now appending rather than overwriting data. The copy data activi...
Richard_100
Resolver I
Resolver IYes, I discovered this today. Can confirm that changing the JSON to Overwrite from OverwriteSchema works.
For me, it applies only to lakehouse-to-lakehouse Copy Data activities.
I have lots of Copy Data pipelines connected to on-premise DB via a gateway and I've just tested one and the overwriting action works correctly despite the JSON code being "OverwriteSchema".
However, I've just set up a new Copy Data to copy a table from one lakehouse to another, i.e. totally internal to Fabric, and this appends rather than overwrites as per OP's post. I change "OverwriteSchema" to "Overwrite" in the JSON and it works correctly again.
It's like there's a disconnect between the UI and the JSON. This is a dreadful bug.