Forum Discussion

Mimi_98's avatar
Mimi_98
Regular Visitor
2 months ago
Solved

Missing Table from LakeHouse

Hi, I have a question.

Yesterday, I developed several tables for the Silver and Gold layers, created the semantic model, and added all the necessary measures. However, when I came back today to continue my work, I found that all of those tables were gone.

Has anyone experienced this before? If so, could you share what caused it?

For context, the notebooks were only run manually and were not scheduled, as they are still under development.

  • Hi Mimi_98 ,

     

    This is a common frustration, but in most cases the data isn't actually lost — it's just not where you expect it to be. Here's what I'd check, roughly in order of likelihood.


    First, verify the default Lakehouse in your notebook. This is the number one cause I've seen in this community. When you call .saveAsTable(), it writes to whatever Lakehouse is set as the default in the notebook (bottom-left of the notebook UI), not necessarily the one you're browsing later. If you had multiple Lakehouses attached, your tables may be sitting in a different one entirely. Open the notebook you ran yesterday and confirm which Lakehouse it was actually pointed at.


    Second, check whether the data exists on disk even if the tables aren't registered. Open the Lakehouse you think the tables should be in, and look at both the Tables and Files sections in the explorer. Then run this in a new notebook attached to that Lakehouse:

    SHOW TABLES; --SQL Command
    
    mssparkutils.fs.ls("Tables/") -- Python Command


    If SHOW TABLES returns nothing but mssparkutils.fs.ls("Tables/") shows folders with _delta_log inside them, your data is physically there — it just didn't get registered in the metastore. This typically happens when code uses .save("Tables/my_table") (path-based write) instead of .saveAsTable("my_table") (catalog-registered write). You can fix it by re-registering:

    spark.sql("CREATE TABLE my_table USING DELTA LOCATION 'Tables/my_table'")
    1. A few other things worth ruling out:
      If you saved as Parquet instead of Delta, the tables won't appear in the Tables section or the SQL endpoint. Only Delta format gets tracked in the catalog.
    2. If your Lakehouse has schema support enabled (you'd see a dbo schema under Tables), then .save("Tables/...") bypasses schema registration and creates "Unidentified" entries. You need to use .saveAsTable("dbo.my_table") instead.
    3. Review your notebook code for any DROP TABLE IF EXISTS or .mode("overwrite") patterns. With managed tables, DROP TABLE removes both the metadata and the underlying files — so if your notebook drops and recreates tables as part of its flow and something interrupted the recreation step, the data would genuinely be gone.
    4. The SQL analytics endpoint has a background metadata sync that can lag behind the actual Lakehouse state, especially on lower-tier capacities. Try opening the SQL endpoint and hitting Refresh, or run REFRESH TABLE table_name in a notebook.

    To help narrow it down further — when you say the tables are "gone," are they missing from the Lakehouse explorer, the SQL endpoint, the semantic model, or all three? And how exactly did your notebook write them (.saveAsTable(), .save() with a path, or Spark SQL CREATE TABLE)? That detail would point to the exact cause pretty quickly.

4 Replies

  • v-menakakota's avatar
    v-menakakota
    Community Support

    Hi Mimi_98 ,


    Thank you for contacting the Microsoft Fabric community forum

    Could you please check if you're in the correct workspace and Lakehouse?

    Also, can you confirm:

    • How the tables were created (Notebook, Dataflow, Pipeline, etc.)?
    • Whether the data files are still visible in the Files section of the Lakehouse?

    Additionally, please review the notebook run history to see if any operation may have overwritten or removed the tables.

    Sharing these details will help us investigate the issue further.

     

    Best Regards, 
    Community Support Team

  • Hi Mimi_98 ,

     

    This is a common frustration, but in most cases the data isn't actually lost — it's just not where you expect it to be. Here's what I'd check, roughly in order of likelihood.


    First, verify the default Lakehouse in your notebook. This is the number one cause I've seen in this community. When you call .saveAsTable(), it writes to whatever Lakehouse is set as the default in the notebook (bottom-left of the notebook UI), not necessarily the one you're browsing later. If you had multiple Lakehouses attached, your tables may be sitting in a different one entirely. Open the notebook you ran yesterday and confirm which Lakehouse it was actually pointed at.


    Second, check whether the data exists on disk even if the tables aren't registered. Open the Lakehouse you think the tables should be in, and look at both the Tables and Files sections in the explorer. Then run this in a new notebook attached to that Lakehouse:

    SHOW TABLES; --SQL Command
    
    mssparkutils.fs.ls("Tables/") -- Python Command


    If SHOW TABLES returns nothing but mssparkutils.fs.ls("Tables/") shows folders with _delta_log inside them, your data is physically there — it just didn't get registered in the metastore. This typically happens when code uses .save("Tables/my_table") (path-based write) instead of .saveAsTable("my_table") (catalog-registered write). You can fix it by re-registering:

    spark.sql("CREATE TABLE my_table USING DELTA LOCATION 'Tables/my_table'")
    1. A few other things worth ruling out:
      If you saved as Parquet instead of Delta, the tables won't appear in the Tables section or the SQL endpoint. Only Delta format gets tracked in the catalog.
    2. If your Lakehouse has schema support enabled (you'd see a dbo schema under Tables), then .save("Tables/...") bypasses schema registration and creates "Unidentified" entries. You need to use .saveAsTable("dbo.my_table") instead.
    3. Review your notebook code for any DROP TABLE IF EXISTS or .mode("overwrite") patterns. With managed tables, DROP TABLE removes both the metadata and the underlying files — so if your notebook drops and recreates tables as part of its flow and something interrupted the recreation step, the data would genuinely be gone.
    4. The SQL analytics endpoint has a background metadata sync that can lag behind the actual Lakehouse state, especially on lower-tier capacities. Try opening the SQL endpoint and hitting Refresh, or run REFRESH TABLE table_name in a notebook.

    To help narrow it down further — when you say the tables are "gone," are they missing from the Lakehouse explorer, the SQL endpoint, the semantic model, or all three? And how exactly did your notebook write them (.saveAsTable(), .save() with a path, or Spark SQL CREATE TABLE)? That detail would point to the exact cause pretty quickly.

    • v-menakakota's avatar
      v-menakakota
      Community Support

      Hi Mimi_98 ,

      I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you .

       

      Best Regards, 
      Community Support Team

      • v-menakakota's avatar
        v-menakakota
        Community Support

        Hi @Mimi_98 ,

        I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you .

         

        Best Regards, 
        Community Support Team