Read Lakehouse A, write Lakehouse B, using Python only (not Spark)

Hi Arnaudlebet,

Microsoft Fabric currently allows you to set only one default Lakehouse for your notebook session, which means all standard path-based operations (like simple Pandas file reads/writes) are directed to this default Lakehouse. However, there are methods to work with data across multiple Lakehouses within a single notebook in Python-only environments, though they do require some manual handling of file system paths.

You can add more than one Lakehouse to your notebook via the Lakehouse Explorer pane. However, only one Lakehouse can be set as "default" at a time, which controls where relative file paths operate.

o work directly with files from different Lakehouses in the same notebook, use the ABFS (Azure Blob File System) absolute path for the files you want to read from or write to in the non-default Lakehouse. You copy this path from the Lakehouse Explorer or file context menu.

When using Pandas (not Spark), you can reference these ABFS paths directly in your code when reading or writing files. For example:

import pandas as pd

# Read from a configuration file in Lakehouse A using its ABFS path
config_df = pd.read_csv('abfss://root@<lakehouse_A_id>.dfs.fabric.microsoft.com/Files/config.csv')

# Process your data as required...

# Write result to Lakehouse B using its ABFS path
result_df.to_csv('abfss://root@<lakehouse_B_id>.dfs.fabric.microsoft.com/Files/result.csv')

Substitute <lakehouse_A_id> and <lakehouse_B_id> with your actual Fabric Lakehouse resource IDs.

Also check this thread: https://community.fabric.microsoft.com/t5/Fabric-platform/Switching-Lakehouses-in-a-notebook/m-p/3845685

Hope that helps.

Onur