baguena_raf
Advocate I
Advocate IDelta table statistics, maintenance and properties configuration
This notebook serves as an example of the possible implementation and subsequent automation of audit and maintenance operations on the tables in our lakehouse.
It extends functionality and allows you to streamline maintenance operations available from the lakehouse explorer and schedule them as needed, as well as check the evolution of the tables residing in it and the configuration of their properties.
In this way, we achieve a more efficient, robust, and complete ecosystem.
You will need to:
- Download and import the notebook to a Fabric capacity backed workspace
- Attach your own lakehouse to the notebook in order to test and try the samples provided
- Configure the specified variables as you need them before running, specially the properties setting section
The notebook is divided into different cells for each section, including comments and type hinting.
The code does not modify your own data. However it will delete unused Delta table files if the VACUUM operation is executed, read everything first before executing anything.
Feel free to adapt the samples for your own needs, hope you find it useful.
Credits & acknowledgments
- Sandeep Pawar for its base analysis for Delta table stats
- Miles Cole for its brilliant article about table compaction
Notebook is available here.
No RepliesBe the first to reply