Forum Discussion
Delta Lake VACUUM error
I have a table I daily overwrite from a DFg2. I have been doing this overwrite for several days, so it generates a new parquet file every day, and this overwrite is added in the metadata. I understand that with the VACUUM command I can delete those parquet files that are not used since X days ago. I guess my old parquet files are not used because what I do is to overwrite the whole table. I clicked on Run VACUUM command using retention threshold 7 days, and it didn't work. In fact, I got an error:
Executing vacuum with retention hours: 168.0
An error occurred while calling o313.vacuum.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 24 in stage 28.0 failed 4 times, most recent failure: Lost task 24.3 in stage 28.0 (TID 980) (vm-bb589657 executor 1): java.lang.AssertionError: assertion failed
How does VACUUM command work? Why isn't it working for my very simple use case?
I've deleted the table, probably internal issues with the tables, because with other tables I can run VACUUM command.
5 Replies
Hi amaaiia ,
Thanks for using Fabric Community.
Did you got a chance to look into this doc?
Use table maintenance feature to manage delta tables in Fabric - Microsoft Fabric | Microsoft Learn
Can you please share the screenshot of the issue you are facing?Hi amaaiia ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others .
Otherwise, will respond back with the more details and we will try to help .Hi amaaiia ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others .
Otherwise, will respond back with the more details and we will try to help .
I've deleted the table, probably internal issues with the tables, because with other tables I can run VACUUM command.
Hi amaaiia ,
It was great to know that you were able to get to a resolution . We expect you to keep using this forum and also motivate others to do that same . You can always help other community members by answering to their queries
