topic Re: Python Notebook read Delta Table using default lakehouse in Data Engineering

Python Notebook read Delta Table using default lakehouse

MangoMagic — Fri, 11 Jul 2025 17:51:32 GMT

In python notebook is there a way to read delta table from default lakehouse which is already mounted instead of using absolute path with workspace?

Would prefer to use relative paths, the code snippets for writing to lakehouse table have this option but when reading from table the code snippet seems to only have option full path 😕

Re: Python Notebook read Delta Table using default lakehouse

Vinodh247 — Sat, 12 Jul 2025 14:11:27 GMT

Yes, you can read Delta tables in Microsoft Fabric Notebooks using relative paths, similar to how you write them. The key is to reference the mounted Lakehouse path, typically under /lakehouse/default (for the default Lakehouse).

To Read a Delta Table from the Default Lakehouse (Mounted):

If the Lakehouse is already mounted in the notebook (as it usually is) you can use a relative path like

from deltalake import DeltaTable # Relative path to the delta table inside the default lakehouse relative_table_path = "/lakehouse/default/Files/tables/my_table" # Read delta table dt = DeltaTable(relative_table_path) df = dt.to_pyarrow_dataset().to_table().to_pandas() display(df)

Do not use the full Fabric workspace URL (abfss://...) unless required for advanced use cases or cross-Lakehouse access. For Tables instead of Files, use /lakehouse/default/Tables/my_table if that is where your Delta tables are written.

Re: Python Notebook read Delta Table using default lakehouse

MangoMagic — Sat, 12 Jul 2025 14:37:05 GMT

Thanks, for this delta table why does it need to have several intermediate functions to load into a dataframe:

dt.to_pyarrow_dataset().to_table().to_pandas()

Is there a better way to read delta table into a data frame to manipulate in Python notebook?

What is "to_pyarrow_dataset"?

Re: Python Notebook read Delta Table using default lakehouse

Vinodh247 — Sun, 13 Jul 2025 06:18:13 GMT

yeah you can do this as well & convert to pandas if needed

df = spark.read.format("delta").load("/lakehouse/default/Tables/my_table")
df.show()

The reason why pyarrow was used:

to_pyarrow_dataset()
Loads the Delta table as a PyArrow dataset, which is a fast, columnar format. This enables efficient filtering, column pruning, and scanning of large data volumes.
.to_table()
Converts the Arrow dataset to an in-memory Arrow Table.
.to_pandas()
Finally, converts the Arrow Table to a Pandas DataFrame.

Why not just dt.to_pandas()?

Because:

The Delta Lake Python bindings (delta-rs) are optimized for interoperability with PyArrow, not Pandas.
They expect users to control the intermediate stages, for ex: filtering data before loading it into memory via Arrow.

Re: Python Notebook read Delta Table using default lakehouse

v-veshwara-msft — Tue, 15 Jul 2025 10:39:08 GMT

Hi @MangoMagic ,

Thanks for raising this in Microsoft Fabric Community.

Yes, you can read Delta tables using relative paths if your notebook is attached to the default Lakehouse. The Tables/YourTableName path works without needing to specify the full workspace or Lakehouse path.

For example by using this code

df = spark.read.format("delta").load("Tables/Sales")

Output:

This works as long as the table exists under the Tables folder of the Lakehouse mounted to the current notebook.

The full ABFS path is only required if you're accessing data from a different Lakehouse. You can copy the paths using the right-click context menu of the Tables in Lakehouse Explorer.

Load data into your lakehouse with a notebook - Microsoft Fabric | Microsoft Learn

Hope this helps. Please reach out for further assistance.

Thank you.

Re: Python Notebook read Delta Table using default lakehouse

v-veshwara-msft — Mon, 21 Jul 2025 04:49:29 GMT

Hi @MangoMagic ,

Just checking in to see if you query is resolved and if any responses were helpful.
Otherwise, feel free to reach out for further assistance.

Thank you.

Re: Python Notebook read Delta Table using default lakehouse

v-veshwara-msft — Fri, 25 Jul 2025 04:47:31 GMT

Hi @MangoMagic
We wanted to kindly follow up regarding your query. If you need any further assistance, please reach out.
Thank you.

Re: Python Notebook read Delta Table using default lakehouse

v-veshwara-msft — Wed, 30 Jul 2025 05:55:16 GMT

Hi @MangoMagic ,
Just wanted to check if the response provided was helpful. If further assistance is needed, please reach out.
Thank you.