Lakehouse Delta Table Bug

Hi AnthonySottile,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for your issue worked? or let us know if you need any further assistance here?

Thanks,

Prashanth Are

MS Fabric community support

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query

hi AnthonySottile,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for your issue worked? or let us know if you need any further assistance here?

Thanks,

Prashanth Are

MS Fabric community support

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query

Hi AnthonySottile,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for your issue worked? or let us know if you need any further assistance here?

Thanks,

Prashanth Are

MS Fabric community support

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query

Hi AnthonySottile ,

Yes, this is a known issue that can occur in Fabric Lakehouse environments when there's a mismatch between the metadata layers used by Spark and the SQL endpoint.

Here's what's likely happening:

• The SQL endpoint uses its own metadata catalog and can often access tables even if the Spark engine has trouble resolving them.
• The Spark engine (used when loading via notebook > Load Data > Spark) relies on the Delta Lake transaction log and metadata files. If these are corrupted, missing, or out of sync, Spark may not recognize the table.

Possible Fixes:

1. Refresh the Lakehouse metadata
   Try restarting the Lakehouse or reloading the workspace. Sometimes metadata caching causes this issue.

2. Check for _delta_log integrity
   Navigate to the Lakehouse file explorer and ensure the _delta_log folder exists and is intact for that table.

3. Re-register the table
   If the table was manually created or altered, try dropping and recreating the table using Spark SQL to ensure it's properly registered in the Spark catalog.

4. Use spark.read.format("delta") manually
   In your notebook, try loading the table manually using:
   
   

df = spark.read.format("delta").load("Tables/YourTableName")

Hi burakkaragoz,

as mentioned this is known issue I’d encourage you to submit your detailed feedback and ideas via Microsoft's official feedback channels, such as the Microsoft Fabric Ideas 

Feedback submitted here is often reviewed by the product teams and can lead to meaningful improvement.

Thanks,

Prashanth