Allow excluding columns from update/change detection in Upsert of Copy Activity

[Yazdan]

Hi All,

Description:
Currently, the Upsert capability in Copy Activity in Microsoft Fabric is very powerful, allowing us to ingest data from multiple sources like SQL Server or Oracle DB and write to Delta tables in a Lakehouse. Using Upsert for CDC scenarios works great and provides excellent performance.

However, in some cases, we need to exclude certain columns from update/change detection in the merge operation. For example, columns like LOAD_DATE, UPDATE_DATE, or other audit/metadata fields are refreshed on every ingestion, and we do not want these columns to trigger unnecessary updates.

It would be extremely useful if Copy Activity Upsert allowed specifying “exclude columns” for change detection, so these fields could be ignored during merge comparisons. While this can be implemented in notebooks, the built-in Upsert capability in Copy Activity is more efficient and avoids the need for extra notebook stages, providing excellent performance.

Adding an exclude columns option would make Upsert in Copy Activity even more powerful and flexible for CDC scenarios.

Request:
Please provide an option to exclude specific columns from update/change detection when using Upsert in Copy Activity.

Regards,

Yazdan Aghajanian