Forum Discussion

jbreunung's avatar
jbreunung
Frequent Visitor
2 years ago
Solved

Change default Lakehouse with notebookutils

We are currently trying to find a solution for our development process. If we use git with branches to transport notebooks between environments, the default lakehouse information is still pointing t...
data engineering
Notebook
  • abm_chris's avatar
    abm_chris
    2 years ago

    Yeah I understand what you're saying but once you've configured it once it's not a problem, just have to remember to do it on the first time you add the notebook to the 'prod' workspace.

    It's just a flat json file stored in OneLake so any config settings I need are hidden away from users. I just use the below to get the name of the current notebook which will map to one of the elements in the file.

    nb_name = mssparkutils.runtime.context['currentNotebookName']
abm_chris's avatar
abm_chris
Advocate I
2 years ago

Using Deployment pipelines you can specify which the default lakehouse is.

I have mine setup like the below:
Dev workspace connected to source control
Test workspace connected to main branch and sync changes in

Prod workspace not connected to source control

 

I then have deployment pipelines connected from Test to Prod workspace where I can configure default Lakehouse.

 

This is what I have found works for me, I also use a config file on the prod workspace so that I can connect to other Lakehouses' that are not the default one and may be in  a different workspace using the abfss path.

 

 

jbreunung's avatar
jbreunung
Frequent Visitor
2 years ago

Thank you for the comment. I have played with deploymen pipelines as well but I did not like that I have to specifiy the deplyoment rule for each notebook. 

Regarding the config file: Is this something fabric specific? Or do you just have a flat file with the information stored on onelake?