Forum Discussion
Runtime Lakehouse Switch between multiple notebooks
Hi hareshparab-408,
You can try below dynamic options also to achive your desired workaround.1) Don't switch attachments; use full onelake paths
Keep your lh_common attached for the config and use dynamic paths to read/write for the client data for example,
client_id = "1" # from config table
lh_path = f"abfss://{workspace_id}@onelake.dfs.fabric.microsoft.com/{lakehouse_id}/Tables/YourTable"
df = spark.read.format("delta").load(lh_path)Or use the SQL endpoint / Spark.SQL
SELECT * FROM 'ws1'.'lh_1'.'dbo'.'YourTable'
2) Create parameterized pipeline for each run.
Create commom Data pipeline in the workspace you want e.g ws-common
- Read config from the lh_common
- For client data processing use same notebook activity with parametrs such as workspace_name, lakehouse_name,table_path
- Notebook will pick the update parameters value and execute by build path dynamically (As same as above option)
I hope this is what you need, please consider as an accepted solution if helps or give some kudos.
Thanks
Hi hareshparab-408,
Another option is to use the 4 part identifier to reference the lakehouse, this should work regardless of the lakehouse context you have (as long as you have a context)
use `workspace name`.lakehouse.schema.table
use `` if your workspace name has spaces.
There is also another option where you don't have to attach a Lakehouse to a notebook, but when you do that you then have to specify the workspace and lake house goods which can be parameter driven in order to write successfully to the lake house tables using the ABFSS path.
# Using GUIDs (most reliable)
abfss_path = "abfss://<workspace_id>@onelake.dfs.fabric.microsoft.com/<lakehouse_id>/Tables/my_table"
# Or using names (if no special chars/spaces)
abfss_path = "abfss://[email protected]/MyLakehouse.Lakehouse/Tables/my_table"