Forum Discussion
Unity Catalog shortcuts in Databricks checkpoint error in Fabric.
I'm having an issue with UC tables in Fabric where about a third of my tables are not showing up past the lakehouse due to this error:
Delta table 'sanction_factpromotions' with '_last_checkpoint' file references version '0' but the associated checkpoint file '00000000000000000000.checkpoint.parquet' is missing from the _delta_log directory. Please review and correct the '_last_checkpoint' file such that it conforms to the Delta specification.
However the _last_checkpoint file indicates the path to the corresponding file but Fabric completely ignores that. Because of this, 1/3 of my tables are not usable in Fabric due to a DeltaTableLastCheckpointedVersionMissing error. Is there a workaround for this?
Hi MWTai,
Thank you for reaching out to Microsoft Fabric Community Forum.
The error suggests that the checkpoint file referenced is missing or not recognized by Fabric, even though it exists in Databricks.
To resolve this, first, check the directory in Databricks and verify whether the missing checkpoint file exists. If it is missing, you may need to regenerate it using Databricks commands.
If the file exists but Fabric is not recognizing it, try deleting and recreating the Unity Catalog shortcut in Fabric. You can also trigger a metadata refresh in Fabric SQL or overwrite the table with new data in Databricks to generate a fresh checkpoint.
Another possible cause is a Lakehouse sync issue in Fabric. Ensure that the Unity Catalog shortcut is properly configured and that all necessary permissions are granted.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Regards,
Vinay Pabbu
7 Replies
Hi MWTai,
Thank you for reaching out to Microsoft Fabric Community Forum.
The error suggests that the checkpoint file referenced is missing or not recognized by Fabric, even though it exists in Databricks.
To resolve this, first, check the directory in Databricks and verify whether the missing checkpoint file exists. If it is missing, you may need to regenerate it using Databricks commands.
If the file exists but Fabric is not recognizing it, try deleting and recreating the Unity Catalog shortcut in Fabric. You can also trigger a metadata refresh in Fabric SQL or overwrite the table with new data in Databricks to generate a fresh checkpoint.
Another possible cause is a Lakehouse sync issue in Fabric. Ensure that the Unity Catalog shortcut is properly configured and that all necessary permissions are granted.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Regards,
Vinay PabbuIt didn't work for me.
Actually, I noticed my error came from v2checkpoint which is currently not supported correctly.
There is an idea. Please vote for this if you encounter this issue.
Hi MWTai,
As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
If our response addressed, please mark it as Accept as solution and click Yes if you found it helpful.Regards,
Vinay PabbuHi MWTai,
May I ask if you have gotten this issue resolved?
If it is solved, please mark the helpful reply or share your solution and accept it as solution, it will be helpful for other members of the community who have similar problems as yours to solve it faster.
Regards,
Vinay PabbuHi @MWTai,
As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
If our response addressed, please mark it as Accept as solution and click Yes if you found it helpful.Regards,
Vinay Pabbu
I have exactly the same issue. I don't know what to do, everything was working fine and suddenly we got this issue on each table. They are all shortcut tables on an external location. The tables are filled by databricks. I can do select queries though. The issue is in the lakehouse, analytical end point and semantic model. Consequence is the semantic model can't be refreshed. I generated new checkpoints by doing more than 10 operations, however, I still have the issue. All the files are there, I tried redoing the shortcut, same issue. Could you please help?
I even deleted the _last_checkpoint file and did "
import org.apache.spark.sql.delta.DeltaLogDeltaLog.forTable(spark,"Tables/silver/table_name").checkpoint()", same issue after.
