Forum Discussion

mlukicic's avatar
mlukicic
Regular Visitor
10 months ago

Lakehouse Tutorial issues preparing and transforming data in the lakehouse

Hey Everyone,    I am following the End-to-end Lake House Tutorial under Lakehouse tutorial - Prepare and transform lakehouse data - Microsoft Fabric | Microsoft Learn .   When running the second...
data engineering
lakehouse
notebook
mlukicic's avatar
mlukicic
Regular Visitor
10 months ago

Hi!

Thank you for your suggestion.

wwilakehouse.dbo.fact_sale raises the same AnalysisException: [TABLE_OR_VIEW_NOT_FOUND] Exception.

 

I can read the parquet file with:

df = spark.read.format("parquet").load('Files/wwi-raw-data/full/fact_sale')

However, I would like to read the Delta files (like tutorial does).

 

I noticed two things. First, in Explorer, under the wwilakehouse / Tables, I have Undefined folders under each dimension_ table.

 

And second, running:

spark.sql("SELECT current_catalog()").show(truncate=False)
spark.sql("SELECT current_database()").show(truncate=False)

will return:

+-----------------+
|current_catalog()|
+-----------------+
|spark_catalog |
+-----------------+

+----------------------------------------------------------------------------------------+
|current_database() |
+----------------------------------------------------------------------------------------+
|chimcobldhq2ahj1c9p6iop0dhgmmpb8dtqn6p90ahqn8rrid5gmo81d410k29bnetkmoobbclk6utbjckim8ojf|
+----------------------------------------------------------------------------------------+

Maybe I am interpreting it wrong, but it seems to me like spark is not attached to the lakehouse (even the Notebook is).

 

Thanks,
Marko

v-aatheeque's avatar
v-aatheeque
Community Support
9 months ago

Hi mlukicic 

It looks like your Spark session isn’t properly connected to the Lakehouse, which is why the table reference fails. The output showing spark_catalog and a long default database name means Spark is running in a default workspace context, not inside the Lakehouse.

 

Try this suggestion points as below :

  • Go to the notebook toolbar.
  • Look for the “Lakehouse” icon or “Attached items.”
  • Click “Attach to” and select your Lakehouse :  wwilakehouse.
  • Wait until it shows up under “Attached items” in the Explorer panel on the left.

Once attached, re-run your queries and it should be able to access the Lakehouse tables correctly.

Hope this helps !!
Thank You.

  • mlukicic's avatar
    mlukicic
    Regular Visitor
    9 months ago

    Hi v-aatheeque!

     

    Thank you for your reply. Can you help me locate the "Lakehouse" icon or "Attached items"?

     

     

    Am I missing something? 

     

    Thank you very much for your help,

     

    Marko

    • v-aatheeque's avatar
      v-aatheeque
      Community Support
      9 months ago

      Hi mlukicic 

      Based on the screenshot you shared, I followed the steps from the provided document (Lakehouse tutorial - Prepare and transform lakehouse data - Microsoft Fabric | Microsoft Learn) and was able to get the expected output as shown below.

      Can you once cross check this below steps :

      1. Refresh your Lakehouse view to make sure the tables (fact_sale, dimension_date, and dimension_city) are actually there.

      2. Check that your Spark code is using the right schema.

      3. Make sure your notebook is connected to the right Lakehouse.

      4. Double-check the table names, even a small spelling mistake can cause this error.

      To attach the lakehouse as per the newFabric UI :

      • In the Explorer panel, click + Add data items.
      • Select New Lakehouse/Existing data sources.
      • Choose your Lakehouse (e.g., wwilakehouse) and click Add.
      • Ensure it appears under Items with a database icon.

      Hope this helps !!
      Thank You.

       

       

      • v-aatheeque's avatar
        v-aatheeque
        Community Support
        9 months ago

        Hi mlukicic 

        Have you had a chance to look through the responses shared earlier? If anything is still unclear, we’ll be happy to provide additional support.