Forum Discussion

Mauro89's avatar
Mauro89
Super User
5 months ago
Solved

Delta Table time travel not readable

Hi folks,   I face some challenges if I want to get a certain delta table version via the following command:   ver = 9 deltaTable = "Tables/xyz" spark.sql(f"SELECT * FROM delta.`{deltaT...
lakehouse
notebook
  • V-yubandi-msft's avatar
    V-yubandi-msft
    5 months ago

    Thank you for sharing the screenshot. When I refer to an unbroken sequence, I mean that the transaction log versions should increase consecutively without any gaps, such as 3, 4, 5, 6, etc. The sequence in your screenshot appears consistent.

     

    Missing files like 00000000000000000000.json, 00000000000000000001.json, and 00000000000000000002.json are not necessarily a problem, since Delta can remove older log files based on log retention settings.

     

    With a checkpoint at version 10, Spark may only be able to reconstruct the table starting from that checkpoint if earlier logs needed for version 9 are no longer available. In this case, time travel would start from version 10.

     

    Thanks....

YassineHachguer's avatar
YassineHachguer
Regular Visitor
5 months ago

It could be related to log retention or checkpointing. Even if older files exist in _delta_log, they may not be usable for time travel if the checkpoint or metadata chain starts at version 10. You might want to check the table history with DESCRIBE HISTORY to confirm which versions are actually available.