Forum Discussion
Delta Table time travel not readable
Thank you for sharing the screenshot. When I refer to an unbroken sequence, I mean that the transaction log versions should increase consecutively without any gaps, such as 3, 4, 5, 6, etc. The sequence in your screenshot appears consistent.
Missing files like 00000000000000000000.json, 00000000000000000001.json, and 00000000000000000002.json are not necessarily a problem, since Delta can remove older log files based on log retention settings.
With a checkpoint at version 10, Spark may only be able to reconstruct the table starting from that checkpoint if earlier logs needed for version 9 are no longer available. In this case, time travel would start from version 10.
Thanks....
Hi V-yubandi-msft,
thanks for your input. And indeed there is a checkpoint at 10.
When you say "form an unbroken sequence", what do you mean by that?
Is it as shown in the image ment to be "broken" as there is no JSON 1 and 2?
Thanks!
Thank you for sharing the screenshot. When I refer to an unbroken sequence, I mean that the transaction log versions should increase consecutively without any gaps, such as 3, 4, 5, 6, etc. The sequence in your screenshot appears consistent.
Missing files like 00000000000000000000.json, 00000000000000000001.json, and 00000000000000000002.json are not necessarily a problem, since Delta can remove older log files based on log retention settings.
With a checkpoint at version 10, Spark may only be able to reconstruct the table starting from that checkpoint if earlier logs needed for version 9 are no longer available. In this case, time travel would start from version 10.
Thanks....