Adding a prescript in a copy job in Fabric

[Ness_]

I've set up a Copy job activity in Microsoft Fabric that reads approximately five tables from Visma and encountered a problem where it does not append, even though I set it as a method, resulting in duplicated data. I solved the problem by reading distinct rows.

What I want to try now is to add a Prescript in the Copy Job activity to truncate the table, but it seems that it's not possible. Has anyone encountered this problem, and how did you solve it?

[Anonymous]

Hi @Ness_ ,

Could you try Incremental mode in copy job?

https://learn.microsoft.com/en-us/fabric/data-factory/create-copy-job#create-a-copy-job

Best Regards,

Wearsky

[rajendraongole1]

Hi @Ness_ - In Microsoft Fabric does not have built-in prescript options within Copy Activity, the best way to handle this is by orchestrating a solution using SQL scripts or other automation tools like Logic Apps or Power Automate. This way, you can ensure your tables are truncated before loading fresh data to prevent duplication.

If you're working in Dataflow Gen2, you can set up steps within the Dataflow that execute a query on the target table to delete all rows before loading new data. However, this solution requires manual setup in dataflows for managing pre-loading processes.

Solved: Truncate Warehouse SQL table - Microsoft Fabric Community

Solved: Right workaround to solve the unsupported Truncate... - Microsoft Fabric Community