ADF Binary Column

Vishal4u — Tue, 30 Jul 2024 14:44:58 GMT

Is there a way we can lookup on Binary Hash/table to determine the data exist or changed records using ADF?

Re: ADF Binary Column

Anonymous — Wed, 31 Jul 2024 02:22:26 GMT

I think you can refer to the following steps to check it out.

1. Choose which data stream you're checking for existence in the Right stream dropdown.

2. Specify whether you're looking for the data to exist or not exist in the Exist type setting.

Load data with computed hash values into a database table.

1. Extract new data and compute hashes.

2. Join new data with existing data based on key columns to compare hashes.

3. Identify new or changed records (where new hash != existing hash).

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Vishal4u — Thu, 01 Aug 2024 16:42:37 GMT

Sounds interesting, but my question was from a azure data factory?