topic Re: Lakehouse Tutorial issues preparing and transforming data in the lakehouse in Data Engineering

Lakehouse Tutorial issues preparing and transforming data in the lakehouse

mlukicic — Fri, 24 Oct 2025 11:28:05 GMT

Hey Everyone,

I am following the End-to-end Lake House Tutorial under Lakehouse tutorial - Prepare and transform lakehouse data - Microsoft Fabric | Microsoft Learn .

When running the second Notebook `02 - Data Transformation - Business Aggregates.ipynb` (step 20 from the https://learn.microsoft.com/en-us/fabric/data-engineering/tutorial-lakehouse-data-preparation) :

df_fact_sale = spark.read.table("wwilakehouse.fact_sale") df_dimension_date = spark.read.table("wwilakehouse.dimension_date") df_dimension_city = spark.read.table("wwilakehouse.dimension_city")

...I get an AnalysisException:

--------------------------------------------------------------------------- AnalysisException Traceback (most recent call last) Cell In[11], line 1 ----> 1 df_fact_sale = spark.read.table("wwilakehouse.fact_sale") 2 df_dimension_date = spark.read.table("wwilakehouse.dimension_date") 3 df_dimension_city = spark.read.table("wwilakehouse.dimension_city") File /opt/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py:484, in DataFrameReader.table(self, tableName) 450 def table(self, tableName: str) -> "DataFrame": 451 """Returns the specified table as a :class:`DataFrame`. 452 453 .. versionadded:: 1.4.0 (...) 482 >>> _ = spark.sql("DROP TABLE tblA") 483 """ --> 484 return self._df(self._jreader.table(tableName)) File ~/cluster-env/trident_env/lib/python3.11/site-packages/py4j/java_gateway.py:1322, in JavaMember.__call__(self, *args) 1316 command = proto.CALL_COMMAND_NAME +\ 1317 self.command_header +\ 1318 args_command +\ 1319 proto.END_COMMAND_PART 1321 answer = self.gateway_client.send_command(command) -> 1322 return_value = get_return_value( 1323 answer, self.gateway_client, self.target_id, self.name) 1325 for temp_arg in temp_args: 1326 if hasattr(temp_arg, "_detach"): File /opt/spark/python/lib/pyspark.zip/pyspark/errors/exceptions/captured.py:185, in capture_sql_exception.<locals>.deco(*a, **kw) 181 converted = convert_exception(e.java_exception) 182 if not isinstance(converted, UnknownException): 183 # Hide where the exception came from that shows a non-Pythonic 184 # JVM exception message. --> 185 raise converted from None 186 else: 187 raise AnalysisException: [TABLE_OR_VIEW_NOT_FOUND] The table or view `wwilakehouse`.`fact_sale` cannot be found. Verify the spelling and correctness of the schema and catalog. If you did not qualify the name with a schema, verify the current_schema() output, or qualify the name with the correct schema and catalog. To tolerate the error on drop use DROP VIEW IF EXISTS or DROP TABLE IF EXISTS.; 'UnresolvedRelation [wwilakehouse, fact_sale], [], false

How can I solve this issue?

Thanks for your help.

Re: Lakehouse Tutorial issues preparing and transforming data in the lakehouse

Thomaslleblanc — Fri, 24 Oct 2025 15:12:56 GMT

try database.schema.table names like wwilakehouse.dbo.fact_sale

or reading Tables\fact_sales

Re: Lakehouse Tutorial issues preparing and transforming data in the lakehouse

mlukicic — Fri, 24 Oct 2025 15:49:59 GMT

Hi!

Thank you for your suggestion.

wwilakehouse.dbo.fact_sale raises the same AnalysisException: [TABLE_OR_VIEW_NOT_FOUND] Exception.

I can read the parquet file with:

df = spark.read.format("parquet").load('Files/wwi-raw-data/full/fact_sale')

However, I would like to read the Delta files (like tutorial does).

I noticed two things. First, in Explorer, under the wwilakehouse / Tables, I have Undefined folders under each dimension_ table.

And second, running:

spark.sql("SELECT current_catalog()").show(truncate=False) spark.sql("SELECT current_database()").show(truncate=False)

will return:

+-----------------+ |current_catalog()| +-----------------+ |spark_catalog | +-----------------+ +----------------------------------------------------------------------------------------+ |current_database() | +----------------------------------------------------------------------------------------+ |chimcobldhq2ahj1c9p6iop0dhgmmpb8dtqn6p90ahqn8rrid5gmo81d410k29bnetkmoobbclk6utbjckim8ojf| +----------------------------------------------------------------------------------------+

Maybe I am interpreting it wrong, but it seems to me like spark is not attached to the lakehouse (even the Notebook is).

Thanks,
Marko

Re: Lakehouse Tutorial issues preparing and transforming data in the lakehouse

v-aatheeque — Tue, 28 Oct 2025 08:50:28 GMT

Hi @mlukicic

It looks like your Spark session isn’t properly connected to the Lakehouse, which is why the table reference fails. The output showing spark_catalog and a long default database name means Spark is running in a default workspace context, not inside the Lakehouse.

Try this suggestion points as below :

Go to the notebook toolbar.
Look for the “Lakehouse” icon or “Attached items.”
Click “Attach to” and select your Lakehouse : wwilakehouse.
Wait until it shows up under “Attached items” in the Explorer panel on the left.

Once attached, re-run your queries and it should be able to access the Lakehouse tables correctly.

Hope this helps !!
Thank You.

Re: Lakehouse Tutorial issues preparing and transforming data in the lakehouse

mlukicic — Tue, 28 Oct 2025 12:38:17 GMT

Hi @v-aatheeque!

Thank you for your reply. Can you help me locate the "Lakehouse" icon or "Attached items"?

Am I missing something?

Thank you very much for your help,

Marko

Re: Lakehouse Tutorial issues preparing and transforming data in the lakehouse

v-aatheeque — Wed, 29 Oct 2025 10:09:06 GMT

Hi @mlukicic

Based on the screenshot you shared, I followed the steps from the provided document (Lakehouse tutorial - Prepare and transform lakehouse data - Microsoft Fabric | Microsoft Learn) and was able to get the expected output as shown below.

Can you once cross check this below steps :

1. Refresh your Lakehouse view to make sure the tables (fact_sale, dimension_date, and dimension_city) are actually there.

2. Check that your Spark code is using the right schema.

3. Make sure your notebook is connected to the right Lakehouse.

4. Double-check the table names, even a small spelling mistake can cause this error.

To attach the lakehouse as per the newFabric UI :

In the Explorer panel, click + Add data items.
Select New Lakehouse/Existing data sources.
Choose your Lakehouse (e.g., wwilakehouse) and click Add.
Ensure it appears under Items with a database icon.

Hope this helps !!
Thank You.

Re: Lakehouse Tutorial issues preparing and transforming data in the lakehouse

v-aatheeque — Fri, 31 Oct 2025 13:13:27 GMT

Hi @mlukicic

Have you had a chance to look through the responses shared earlier? If anything is still unclear, we’ll be happy to provide additional support.

Re: Lakehouse Tutorial issues preparing and transforming data in the lakehouse

mlukicic — Fri, 31 Oct 2025 16:22:31 GMT

Hi @v-aatheeque,

Thank you for your help and time.

Unfortunately, I couldn’t get this to work. I deleted the entire Lakehouse and started over, following the tutorial step by step. However, I noticed some differences between my UI and the tutorial.

When I opened the Notebooks from the tutorial, I saw that some resources were attached in the Explorer, all marked with red icons and a white cross, indicating an error. This happened on both Notebooks. I deleted all these resources, added the wwilakehouse Lakehouse, and set it as the default resource.

The first Notebook ran without any issues. However, in the second Notebook, the files were not auto-registered as Delta tables as described in the tutorial.

Unless you have any suggestions on where to look next, I’m closing this and moving forward. From my perspective as a machine learning engineer, Fabric seems workable, but it still needs improvement.

Thanks,

Marko

Re: Lakehouse Tutorial issues preparing and transforming data in the lakehouse

v-aatheeque — Tue, 04 Nov 2025 11:24:12 GMT

Hi @mlukicic

Thank you for the detailed update ,It’s great to see the thorough steps you’ve taken to validate and troubleshoot the tutorial process.

The differences you noticed in the UI and the auto-registration behavior are valuable observations. Since Fabric is still evolving rapidly, your feedback could help improve future experiences.

If you’d like, you can share this suggestion directly through the Microsoft Fabric Ideas portal (https://ideas.fabric.microsoft.com/). The product team actively reviews user inputs there for feature improvements.

Appreciate your time exploring this and sharing your insights!

Re: Lakehouse Tutorial issues preparing and transforming data in the lakehouse

mlukicic — Tue, 04 Nov 2025 12:02:09 GMT

Dear @v-aatheeque ,

Many thanks. And thank you for the link. In fact, I would like to share a few suggestions, as I believe the strategy is great and could become an excellent tool for machine learning engineers working with graphs as well.

Kind regards,

Marko