Forum Discussion

maxkent's avatar
maxkent
Frequent Visitor
2 years ago
Solved

Recursive notebooks calls default lakehouse

We have the following notebooks:   - Notebook A, with no default lakehouse - Notebook B, with a default lakehouse   Is it an expected scenario that when calling Notebook B from Notebook A, the d...
  • frithjof_v's avatar
    frithjof_v
    2 years ago

    Maybe it can work to use this code in the first cell in Notebook B?

     

     

     

     

    %%configure -f
    {
        "defaultLakehouse": {  
            "name": "<lakehouseName>",
            "id": "<lakehouseID>",
            "workspaceId": "<workspaceID>"
        }
    }

     

     

     

     

    In order to try to force Notebook B to use this default Lakehouse, even when it's called from Notebook A.

     

    (I don't have any knowledge regarding your question: Is it an expected scenario that when calling Notebook B from Notebook A, the default lakehouse is taken from Notebook A (in this case none)? 

    I hope someone else can answer that.)

     

    Maybe these articles are somehow relevant? 

    https://learn.microsoft.com/en-us/fabric/data-engineering/configure-high-concurrency-session-notebooks#session-sharing-conditions

    and

    https://fabric.guru/how-to-attach-a-default-lakehouse-to-a-notebook-in-fabric

     

    However, I don't have experience with calling a notebook from another notebook.

    So I hope someone else can answer 😄

     

     

    Btw, I think you can run Spark SQL in Notebook B even if it doesn't have a default Lakehouse.

    However I think it means you would need to use temporary views in your code. At least that worked for me. Ref. one of the last comments here: https://community.fabric.microsoft.com/t5/Data-Engineering/Mounting-NB-on-the-fly/m-p/4057128#M3222

     

    Or you could maybe use plain PySpark (without using Spark SQL).

  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi maxkent ,

     

    Thanks for the reply from frithjof_v . The code you provided is very good!

     

    I created three Notebooks for testing, where Notebook6 and Notebook8 are associated with a lakehouse, while Notebook9 does not have a default lakehouse.

     

    When I call notebook8 in notebook6 it works fine and when I call notebook9 in notebook6 some error occurs.

    from notebookutils import mssparkutils
    
    result = notebookutils.notebook.run("Notebook 9", 60)
    
    display(result)

     

    Since Notebook A does not have a default lake bin in one piece, this context will be passed to Notebook B, resulting in the error you encountered.

     

    You can use the “%%configure” method frithjof_v mentioned to set up lakehouse for the notebook.

     

    You can also use the non-coding method in the screenshot below to assign a lakehouse context to the notebook.

     

    You can see that it runs well after assigning the lakehouse context.

     

    If you have any other questions please feel free to contact me.

     

    Best Regards,
    Yang
    Community Support Team

     

    If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
    If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!