Forum Discussion

mlukicic's avatar
mlukicic
Regular Visitor
9 months ago

Lakehouse Tutorial issues preparing and transforming data in the lakehouse

Hey Everyone,    I am following the End-to-end Lake House Tutorial under Lakehouse tutorial - Prepare and transform lakehouse data - Microsoft Fabric | Microsoft Learn .   When running the second...
data engineering
lakehouse
notebook
Thomaslleblanc's avatar
Thomaslleblanc
Super User
9 months ago

try database.schema.table names like wwilakehouse.dbo.fact_sale

or reading Tables\fact_sales

  • mlukicic's avatar
    mlukicic
    Regular Visitor
    9 months ago

    Hi!

    Thank you for your suggestion.

    wwilakehouse.dbo.fact_sale raises the same AnalysisException: [TABLE_OR_VIEW_NOT_FOUND] Exception.

     

    I can read the parquet file with:

    df = spark.read.format("parquet").load('Files/wwi-raw-data/full/fact_sale')

    However, I would like to read the Delta files (like tutorial does).

     

    I noticed two things. First, in Explorer, under the wwilakehouse / Tables, I have Undefined folders under each dimension_ table.

     

    And second, running:

    spark.sql("SELECT current_catalog()").show(truncate=False)
spark.sql("SELECT current_database()").show(truncate=False)

    will return:

    +-----------------+
|current_catalog()|
+-----------------+
|spark_catalog |
+-----------------+

+----------------------------------------------------------------------------------------+
|current_database() |
+----------------------------------------------------------------------------------------+
|chimcobldhq2ahj1c9p6iop0dhgmmpb8dtqn6p90ahqn8rrid5gmo81d410k29bnetkmoobbclk6utbjckim8ojf|
+----------------------------------------------------------------------------------------+

    Maybe I am interpreting it wrong, but it seems to me like spark is not attached to the lakehouse (even the Notebook is).

     

    Thanks,
    Marko

    • v-aatheeque's avatar
      v-aatheeque
      Community Support
      9 months ago

      Hi mlukicic 

      It looks like your Spark session isn’t properly connected to the Lakehouse, which is why the table reference fails. The output showing spark_catalog and a long default database name means Spark is running in a default workspace context, not inside the Lakehouse.

       

      Try this suggestion points as below :

      • Go to the notebook toolbar.
      • Look for the “Lakehouse” icon or “Attached items.”
      • Click “Attach to” and select your Lakehouse :  wwilakehouse.
      • Wait until it shows up under “Attached items” in the Explorer panel on the left.

      Once attached, re-run your queries and it should be able to access the Lakehouse tables correctly.

      Hope this helps !!
      Thank You.

      • mlukicic's avatar
        mlukicic
        Regular Visitor
        9 months ago

        Hi v-aatheeque!

         

        Thank you for your reply. Can you help me locate the "Lakehouse" icon or "Attached items"?

         

         

        Am I missing something? 

         

        Thank you very much for your help,

         

        Marko