Forum Discussion
Solved
Cannot reference a Notebook "that attaching" to a different default lakehouse.
I'm running a pipeline starting with a DAG. And getting an error I can't reference a notebook "that attaching to a different default lakehouse". I've tried passing "useRootDefaultLakehouse": True on ...
So I fixed this by adding "useRootDefaultLakehouse": True arg to my dag under my silver script.
"args": {"useRootDefaultLakehouse": True}Larger snippet...{"name": "notebook_silver","path": "Notebook Silver","timeoutPerCellInSeconds": 3600,"dependencies": ["Notebook Bronze"],"args": {"useRootDefaultLakehouse": True},}My notebook uses absolute abfss paths to access the bronze datalake and the silver datalake so I think it should have worked according to the docs."To specify the location to read from, you can use the relative path if the data is from the default lakehouse of your current notebook. Or, if the data is from a different lakehouse, you can use the absolute Azure Blob File"
Thank you for the reply. RunMultiple doesn't work because of dependencies. I did run all of these in sequence and the notebook which loads from the bronze dlh to the silver dlh is the issue.
This error occurs if I use the notebook in the DAG and use runMultiple or if I run the notebook via a single run command. If I just run the notebook in a session without using mssparkutils it completes without error.
How can I make sure both reference to my bronze and silver datalake houses are present in my silver notebook? I have added the bronze dlh to my silver notebook, but the issue is persisting. 😞
So I fixed this by adding "useRootDefaultLakehouse": True arg to my dag under my silver script.
"args": {"useRootDefaultLakehouse": True}Larger snippet...{"name": "notebook_silver","path": "Notebook Silver","timeoutPerCellInSeconds": 3600,"dependencies": ["Notebook Bronze"],"args": {"useRootDefaultLakehouse": True},}My notebook uses absolute abfss paths to access the bronze datalake and the silver datalake so I think it should have worked according to the docs."To specify the location to read from, you can use the relative path if the data is from the default lakehouse of your current notebook. Or, if the data is from a different lakehouse, you can use the absolute Azure Blob File"
Thank you for sharing the solution with us!