VACUUM with less than 1 hour retaintion

[DennesTorres]

Hi,

For testing purposes, is it possible to use VACUUM with less than 1 hour retaintion ? 

In order to do it with a 1 hour retaintion I had to set a spark configuration:

spark.conf.set("spark.databricks.delta.retentionDurationCheck.enabled", "false")

But what if, for tests purpose, I would like to VACUUM with less than 1 hour ?

Kind Regards,

Dennes

[HimanshuS-msft]

Hi @DennesTorres , 
When i tested it , it did not threw any error , but at the same time it did not threw any error . 
Just curiuos as to what is the bigger problem which you are tring to solve ?

Thanks 
Himanshu

[DennesTorres]

Hi,

It worked for me in lakehouses.

I'm concerned how these files would affect the performance. 

Kind Regards,

Dennes

[puneetvijwani · Accepted solution]

HI @DennesTorres  lets try like this ..i have seen some users doing that 

spark.sql("VACUUM bronze_nyc_tlc_green.nycgreentaxi RETAIN 0 HOURS")

This can be the way to vacuum everything that is not in use. PS: I havent tested this , to be honest too tired today ( Just kidding ) 
Note : This is also can be  risky and should only be done in a controlled environment where you are certain that you do not need older versions of the data.

Let me also know if that works for u !