Change default Lakehouse with notebookutils

[jbreunung]

We are currently trying to find a solution for our development process.

If we use git with branches to transport notebooks between environments, the default lakehouse information is still pointing to the dev lakehouse which will then cause notebooks from the prod environment to write data to dev.

I found this post here Solved: Re: How to set default Lakehouse in the notebook p... - Microsoft Fabric Community on how to change the default lakehouse, however this only applies for the current session. The statement can also only be executed once per session which is hindering the development.

I found a new function in the notebookutils help function notebook.updateDefinition. According to the documentation in the function it should do what I want. Unfortunately it is still in preview and not part of the official documentation here Microsoft Spark Utilities (MSSparkUtils) for Fabric - Microsoft Fabric | Microsoft Learn

When I try to use this function I have to provide a name, the other parameters seem optional. However, the content paramter checks for the provided type of dictionary and throws an error if it is None (even though the code seems to make a distinction later on..) 

My question is now: Is there any other way than the %%configure command to let my notebook know it should reference lakhouses in the same workspace it currently resides in?

And does anyone know if the function in notebookutils is actually designed to help with that if it works properly in the future?

[abm_chris]

Using Deployment pipelines you can specify which the default lakehouse is.

I have mine setup like the below:
Dev workspace connected to source control
Test workspace connected to main branch and sync changes in

Prod workspace not connected to source control

I then have deployment pipelines connected from Test to Prod workspace where I can configure default Lakehouse.

This is what I have found works for me, I also use a config file on the prod workspace so that I can connect to other Lakehouses' that are not the default one and may be in  a different workspace using the abfss path.

[OldDogNewTricks]

@abm_chris - Would you be able to expand on how you are accomplishing this, config file allows you to reference other workspaces dynamically?

[jbreunung]

Thank you for the comment. I have played with deploymen pipelines as well but I did not like that I have to specifiy the deplyoment rule for each notebook. 

Regarding the config file: Is this something fabric specific? Or do you just have a flat file with the information stored on onelake?

[abm_chris · Accepted solution]

Yeah I understand what you're saying but once you've configured it once it's not a problem, just have to remember to do it on the first time you add the notebook to the 'prod' workspace.

It's just a flat json file stored in OneLake so any config settings I need are hidden away from users. I just use the below to get the name of the current notebook which will map to one of the elements in the file.

nb_name = mssparkutils.runtime.context['currentNotebookName']