will_wainwright's avatar
will_wainwright
Regular Visitor
1 year ago

Lakehouse Data Quality Framework

 Link to notebook:  NB_Testing_Framework 

 

This notebook lets you automate data quality tests for your Fabric Lakehouse.
Simply add your tests to a YAML config file, and this notebook will run them dynamically using Spark SQL.

 

Test results are saved to a Delta table, which you can use to build Power BI reports or set up data alerts.

 

The framework is lightweight and extendable; making it easy to add new test types or customize validations as your needs grow.

 

No RepliesBe the first to reply