Forum Discussion

mlukicic's avatar
mlukicic
Regular Visitor
9 months ago

Lakehouse Tutorial issues preparing and transforming data in the lakehouse

Hey Everyone,    I am following the End-to-end Lake House Tutorial under Lakehouse tutorial - Prepare and transform lakehouse data - Microsoft Fabric | Microsoft Learn .   When running the second...
data engineering
lakehouse
notebook
v-aatheeque's avatar
v-aatheeque
Community Support
9 months ago

Hi mlukicic 

It looks like your Spark session isn’t properly connected to the Lakehouse, which is why the table reference fails. The output showing spark_catalog and a long default database name means Spark is running in a default workspace context, not inside the Lakehouse.

 

Try this suggestion points as below :

  • Go to the notebook toolbar.
  • Look for the “Lakehouse” icon or “Attached items.”
  • Click “Attach to” and select your Lakehouse :  wwilakehouse.
  • Wait until it shows up under “Attached items” in the Explorer panel on the left.

Once attached, re-run your queries and it should be able to access the Lakehouse tables correctly.

Hope this helps !!
Thank You.

mlukicic's avatar
mlukicic
Regular Visitor
9 months ago

Hi v-aatheeque!

 

Thank you for your reply. Can you help me locate the "Lakehouse" icon or "Attached items"?

 

 

Am I missing something? 

 

Thank you very much for your help,

 

Marko

  • v-aatheeque's avatar
    v-aatheeque
    Community Support
    9 months ago

    Hi mlukicic 

    Based on the screenshot you shared, I followed the steps from the provided document (Lakehouse tutorial - Prepare and transform lakehouse data - Microsoft Fabric | Microsoft Learn) and was able to get the expected output as shown below.

    Can you once cross check this below steps :

    1. Refresh your Lakehouse view to make sure the tables (fact_sale, dimension_date, and dimension_city) are actually there.

    2. Check that your Spark code is using the right schema.

    3. Make sure your notebook is connected to the right Lakehouse.

    4. Double-check the table names, even a small spelling mistake can cause this error.

    To attach the lakehouse as per the newFabric UI :

    • In the Explorer panel, click + Add data items.
    • Select New Lakehouse/Existing data sources.
    • Choose your Lakehouse (e.g., wwilakehouse) and click Add.
    • Ensure it appears under Items with a database icon.

    Hope this helps !!
    Thank You.

     

     

    • v-aatheeque's avatar
      v-aatheeque
      Community Support
      9 months ago

      Hi mlukicic 

      Have you had a chance to look through the responses shared earlier? If anything is still unclear, we’ll be happy to provide additional support.

      • mlukicic's avatar
        mlukicic
        Regular Visitor
        9 months ago

        Hi v-aatheeque,

         

        Thank you for your help and time.

         

        Unfortunately, I couldn’t get this to work. I deleted the entire Lakehouse and started over, following the tutorial step by step. However, I noticed some differences between my UI and the tutorial.

         

        When I opened the Notebooks from the tutorial, I saw that some resources were attached in the Explorer, all marked with red icons and a white cross, indicating an error. This happened on both Notebooks. I deleted all these resources, added the wwilakehouse Lakehouse, and set it as the default resource.

         

        The first Notebook ran without any issues. However, in the second Notebook, the files were not auto-registered as Delta tables as described in the tutorial.

         

        Unless you have any suggestions on where to look next, I’m closing this and moving forward. From my perspective as a machine learning engineer, Fabric seems workable, but it still needs improvement.

         

        Thanks,

        Marko