topic Re: Time travel limit 7 days in Data Engineering

Time travel limit 7 days

frithjof_v — Fri, 31 May 2024 08:54:38 GMT

Hi,

Is it possible to increase the retention limit for time travel above 7 days in Fabric?

Or is 7 days the maximum retention duration for time travel in Fabric?

Thank you 😀

Re: Time travel limit 7 days

v-cboorla-msft — Fri, 31 May 2024 10:12:25 GMT

Hi @frithjof_v

Thanks for using Microsoft Fabric Community.

The 7 days period is the limit across the OneLake for the entire Fabric. You can’t change it as of now (you might be able to do it in the future). The main reason behind this is to keep the size of the data manageable as well as the performance. I do recommend backing up your Delta table files regularly so that in case you need to travel back in time for more than 7 days.

For more details please refer to the below links :

Link 1 : Time Travel in Fabric Warehouse: Query Snapshots of the Data at Any Point in Time - RADACAD

Link 2 : Time travel in Warehouse within Microsoft Fabric (preview) - Microsoft Fabric | Microsoft Learn

Appreciate if you could share the feedback on our Microsoft Fabric Ideas. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

I hope this information helps. Please do let us know if you have any further queries.

Thank you.

Re: Time travel limit 7 days

frithjof_v — Fri, 31 May 2024 12:54:07 GMT

Thank you @v-cboorla-msft!

The Microsoft documentation you linked to is related only to data warehouse.

Is there any Microsoft documentation for time travel in Lakehouse?

Thank you 😀

Re: Time travel limit 7 days

frithjof_v — Fri, 31 May 2024 18:07:25 GMT

The maintenance feature allows me to select a retention threshold of more than 7 days:

Does this mean that there is no 7-day upper limit for time travel retention of Lakehouse tables, but only a 7-day upper limit for Warehouse tables?

Re: Time travel limit 7 days

frithjof_v — Sat, 01 Jun 2024 18:12:31 GMT

I also have a Lakehouse table with many versions (more than 1000 versions).

I am still able to query the first version of this table from a Fabric Notebook (SELECT * FROM tableName VERSION AS OF 1), and this version is more than 1 month old.

Does this mean that Fabric Lakehouse tables can have unlimited time travel?

While Fabric Data Warehouse tables are restricted to 7 days time travel.

Re: Time travel limit 7 days

LeonardoBenitez — Fri, 07 Jun 2024 08:14:06 GMT

Thanks for the clear answer 😀
Do you have some documentation link stating the 7 days limit for the lakehouse?
Both links you provided refer to the warehouse experience.