Forum Discussion
Delta table maintenance - is it managed?
Hi frithjof_v
Yes, you’re correct in your understanding of the differences between Lakehouse and Data Warehouse in terms of time travel retention and vacuuming.
Lakehouse:
In Lakehouse, the vacuuming process is part of the Table Maintenance feature. This feature includes the Vacuum operation, which removes old files no longer referenced by a Delta table log. The files need to be older than the retention threshold, and the default file retention threshold is seven days.
Use table maintenance feature to manage delta tables in Fabric - Microsoft Fabric | Microsoft Learn
If you don’t run any maintenance, the Lakehouse tables will keep accumulating all the data until the Vacuum operation is executed. So, if you don’t choose to run vacuuming, then the Lakehouse tables will never be vacuumed.
Data Warehouse:
In Data Warehouse, the data retention for time travel queries is seven days. This is a limit across the OneLake for the entire Fabric. You can’t change it as of now. Therefore, Data Warehouse will be limited to 7 days time travel, and Fabric automatically performs the vacuuming without you being able to control it.
Time travel in Warehouse within Microsoft Fabric (preview) - Microsoft Fabric | Microsoft Learn
Hope this helps. Please let me know if you have any further questions.
Thank you Anonymous,
I'm struggling to understand this information: "However, all the delta tables in OneLake have the same retention period."
Let's say I run vacuum on a Lakehouse table. Then I can choose the retention period for that vacuum operation freely. If I run a vacuum operation on another table, then I can choose another retention period for that vacuum operation.
So I would say that all tables don't have the same retention period. The retention period is chosen by me whenever I choose to run a vacuum operation on a single table. Or am I missing something?
Thank you 😀
Hi again frithjof_v
Yes you are right. I have edited the above post. The actual retention period for each table can be set individually based on your requirements when you perform a vacuum operation.
Hope this helps.😊