Forum Discussion
Helper IVcheck accuracy of data load
how is it possible to check/compare the source to the destination?
for example, the data factory pipelines load data from sql server into warehouse tables and now I would like to see if the loaded data is the same as source and report if there are any differences.
thank you
Hi arkiboys2 ,
I think you can join TABLE1 and TABLE2 first, then use filter to get the rows which's type isn't same. Next create a DerivedColumn to generate Type Change column. Finally, use Select transformation to remain the columns you need.
1. Create Join transformation to TABLE1 and TABLE2.
2. Filter the rows which's type isn't same. Expression: source2@Type != source1@Type
3. Add Type Change column. Expression:concat('from ',source1@Type,' to ', source2@Type)
4. Use Select transformation to remain the columns you need.
Then you will see what you want.
Best Regards
Yilong Zhou
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
4 Replies
Hi arkiboys2 ,
I think you can join TABLE1 and TABLE2 first, then use filter to get the rows which's type isn't same. Next create a DerivedColumn to generate Type Change column. Finally, use Select transformation to remain the columns you need.
1. Create Join transformation to TABLE1 and TABLE2.
2. Filter the rows which's type isn't same. Expression: source2@Type != source1@Type
3. Add Type Change column. Expression:concat('from ',source1@Type,' to ', source2@Type)
4. Use Select transformation to remain the columns you need.
Then you will see what you want.
Best Regards
Yilong Zhou
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
arkiboys2hello, thank you for the details. Is it possible you provide the same but for fabric data factory/dataflow gen2 thank you
- arkiboys2
Hi, thank you for the reply, however I am looking for the fabric version.
