compare source to destination

arkiboys2 — Tue, 16 Jul 2024 11:23:01 GMT

hello,

in dataflowgen2 there are two step

one to show the source (sql server table) nd the other to show the table in the warehouse

how can I compare the columns of these two tables and also to compare their column table types?

it seems in ADF dataflow, I can join these two tables and filter on source2@Type != source1@Type

thank you

Re: compare source to destination

Anonymous — Wed, 17 Jul 2024 04:20:27 GMT

Hi arkiboys2 ,

The Copy activity has an Upsert mode which I think would help here. Simple instructions:

1. Create one Copy activity.

2. Set your source database in the Source tab of the Copy activity.

3. Set your target (or sink) database in the Sink tab. Set the mode to Upsert.

4. Specify the interim schema. This is used to create a transient table which holds data during the Upsert.

5. Specify the unique keys for the source and target table in the Key columns section so the Upsert can take place successfully.

Failing that, simply use a Copy activity to land the data into a table in your target database and use a Stored Proc activity to implement your more complicated logic.

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

topic Re: compare source to destination in Pipelines

compare source to destination

Re: compare source to destination