topic Re: Ways can we validation Fabric and redshift tables after migration in Data Engineering

Ways can we validation Fabric and redshift tables after migration

Saadather — Wed, 10 Sep 2025 08:39:10 GMT

Hi all,

we are currently in process of our migration from redshift to fabric warehouse and i would like to know the ways we can do our validation/testing of same tables between source(redshift) and sink(fabric warehouse). And would like to hear from the experts in industry which are the possible ways we can do that?

Thank you!

Re: Ways can we validation Fabric and redshift tables after migration

BalajiL — Wed, 10 Sep 2025 13:18:32 GMT

Saadather

If you are looking testing perspective, plan your test strategy to look for

Schema validation
Row Count check
Hash check / Checksum
Do parallel run testing
Create reconciliation reports

If you are looking for migration aspect look at the below link:

https://learn.microsoft.com/en-us/fabric/data-warehouse/migration-assistant

Also check "Spectral Core tool" is offering redshift migration and validation.

Thanks !

Re: Ways can we validation Fabric and redshift tables after migration

Saadather — Wed, 10 Sep 2025 13:44:57 GMT

hi BalajiL

Thankyou for clarifying, what i am intrested is by which ways i can do the above checks you mentioned i know via python i do that by connecting to both datasources and writing the queries and comparing the result for both.
What i really want to know is there any tool/way which azure/fabric provide to do so or any thrid party which supports that via automated way instead of me writing the sqls comparison for all the tables?

Re: Ways can we validation Fabric and redshift tables after migration

v-sdhruv — Thu, 11 Sep 2025 10:40:39 GMT

Hi Saadather ,

You might want to check -
Data Validations with Great Expectations in MS Fabric - ISE Developer Blog
Great Expectations (GX) can be used to validate your data migration from Amazon Redshift to Microsoft Fabric Warehouse.

You can begin by:

Connect GX to your Redshift and Fabric Warehouse environments.
Define expectations for key tables and columns.
Run validations in parallel

Hope this helps!
Thank You

Re: Ways can we validation Fabric and redshift tables after migration

v-sdhruv — Thu, 18 Sep 2025 07:56:51 GMT

Hi Saadather ,

Just wanted to check if you got a chance to review the suggestions provided and whether that helped you resolve your query?

Thank You

Re: Ways can we validation Fabric and redshift tables after migration

Saadather — Thu, 18 Sep 2025 12:21:28 GMT

Hi v-sdhruv

The solution only works when have to write two seperate expectations and create a script that could compare the results from these two, what i wanted was something like reconsiliation checks that could give me the results all together.

In GX fourm they also mentioned that currently this cross datasource feature is not supported yet..
https://discourse.greatexpectations.io/t/how-to-compare-two-tables-across-different-databases/588

My further RnD for this use case i found these can be achieved by using another tool called SODA which supports such reconsicliation requiremnts.