Forum Discussion

mlukicic's avatar
mlukicic
Regular Visitor
10 months ago

Lakehouse Tutorial issues preparing and transforming data in the lakehouse

Hey Everyone,    I am following the End-to-end Lake House Tutorial under Lakehouse tutorial - Prepare and transform lakehouse data - Microsoft Fabric | Microsoft Learn .   When running the second...
data engineering
lakehouse
notebook
Thomaslleblanc's avatar
Thomaslleblanc
Super User
10 months ago

try database.schema.table names like wwilakehouse.dbo.fact_sale

or reading Tables\fact_sales

mlukicic's avatar
mlukicic
Regular Visitor
10 months ago

Hi!

Thank you for your suggestion.

wwilakehouse.dbo.fact_sale raises the same AnalysisException: [TABLE_OR_VIEW_NOT_FOUND] Exception.

 

I can read the parquet file with:

df = spark.read.format("parquet").load('Files/wwi-raw-data/full/fact_sale')

However, I would like to read the Delta files (like tutorial does).

 

I noticed two things. First, in Explorer, under the wwilakehouse / Tables, I have Undefined folders under each dimension_ table.

 

And second, running:

spark.sql("SELECT current_catalog()").show(truncate=False)
spark.sql("SELECT current_database()").show(truncate=False)

will return:

+-----------------+
|current_catalog()|
+-----------------+
|spark_catalog |
+-----------------+

+----------------------------------------------------------------------------------------+
|current_database() |
+----------------------------------------------------------------------------------------+
|chimcobldhq2ahj1c9p6iop0dhgmmpb8dtqn6p90ahqn8rrid5gmo81d410k29bnetkmoobbclk6utbjckim8ojf|
+----------------------------------------------------------------------------------------+

Maybe I am interpreting it wrong, but it seems to me like spark is not attached to the lakehouse (even the Notebook is).

 

Thanks,
Marko

  • v-aatheeque's avatar
    v-aatheeque
    Community Support
    9 months ago

    Hi mlukicic 

    It looks like your Spark session isn’t properly connected to the Lakehouse, which is why the table reference fails. The output showing spark_catalog and a long default database name means Spark is running in a default workspace context, not inside the Lakehouse.

     

    Try this suggestion points as below :

    • Go to the notebook toolbar.
    • Look for the “Lakehouse” icon or “Attached items.”
    • Click “Attach to” and select your Lakehouse :  wwilakehouse.
    • Wait until it shows up under “Attached items” in the Explorer panel on the left.

    Once attached, re-run your queries and it should be able to access the Lakehouse tables correctly.

    Hope this helps !!
    Thank You.

    • mlukicic's avatar
      mlukicic
      Regular Visitor
      9 months ago

      Hi v-aatheeque!

       

      Thank you for your reply. Can you help me locate the "Lakehouse" icon or "Attached items"?

       

       

      Am I missing something? 

       

      Thank you very much for your help,

       

      Marko

      • v-aatheeque's avatar
        v-aatheeque
        Community Support
        9 months ago

        Hi mlukicic 

        Based on the screenshot you shared, I followed the steps from the provided document (Lakehouse tutorial - Prepare and transform lakehouse data - Microsoft Fabric | Microsoft Learn) and was able to get the expected output as shown below.

        Can you once cross check this below steps :

        1. Refresh your Lakehouse view to make sure the tables (fact_sale, dimension_date, and dimension_city) are actually there.

        2. Check that your Spark code is using the right schema.

        3. Make sure your notebook is connected to the right Lakehouse.

        4. Double-check the table names, even a small spelling mistake can cause this error.

        To attach the lakehouse as per the newFabric UI :

        • In the Explorer panel, click + Add data items.
        • Select New Lakehouse/Existing data sources.
        • Choose your Lakehouse (e.g., wwilakehouse) and click Add.
        • Ensure it appears under Items with a database icon.

        Hope this helps !!
        Thank You.