topic Re: While configuring notebook session from datafactory pipeline, pipeline notebook activity fails in Pipelines

While configuring notebook session from datafactory pipeline, pipeline notebook activity fails

Binit17 — Tue, 12 Mar 2024 19:06:21 GMT

I am following the fabric documentation to configure lakehouse on notebook from pipeline but getting error Configuration should be valid json object. I see an escape character has been added to the parameter that i am passing.

https://learn.microsoft.com/en-in/fabric/data-engineering/author-execute-notebook#parameterized-session-configuration-from-a-pipeline

Re: While configuring notebook session from datafactory pipeline, pipeline notebook activity fails

Binit17 — Wed, 13 Mar 2024 07:07:56 GMT

I realized if we use the same %%configure command in interactive mode we get the attached error as well

Re: While configuring notebook session from datafactory pipeline, pipeline notebook activity fails

Anonymous — Wed, 13 Mar 2024 07:29:22 GMT

Hi Binit17 ,

Thanks for using Fabric Community.
It looks like issue is with the child nodes and for defaultLakehouse is name, id and workspaceId.
You cannot use parameter name and defaultvalue along with defaultLakehouse.

I would like to understand what you are trying to achieve? So that I can guide you better.

For example -

%%configure { "driverCores": { "parameterName": "driverCoresFromNotebookActivity", "defaultValue": 4 } }

Docs to refer - Develop, execute, and manage notebooks - Microsoft Fabric | Microsoft Learn

Hope this is helpful. Please let me know incase of further queries.

Re: While configuring notebook session from datafactory pipeline, pipeline notebook activity fails

Binit17 — Wed, 13 Mar 2024 07:45:38 GMT

Hi Anonymous ,

I saw in the fabric documentation that these configurations can be passed from the pipeline to the notebooks. I have a situation where i have multiple lakehouse in a workspace and i want to execute a notebook to create tables in different lakehouse from a single notebook. For this i want to pass the lakehouse as a parameter from pipeline to notebook so it sets the default lakehouse as per input parameter and create tables in appropriate lakehouses.

Re: While configuring notebook session from datafactory pipeline, pipeline notebook activity fails

Anonymous — Thu, 14 Mar 2024 06:36:11 GMT

Hello Binit17

Apologies for the delay in reply from our side.

You can follow below step inorder to pass them as pipeline parameters -

In Pipeline:

In Notebook:

%%configure { "defaultLakehouse": { // This overwrites the default lakehouse for current session "name": { "parameterName": "paramterName", "defaultValue": "gopi_lake_house_1" } , "id": { "parameterName": "parameterId", "defaultValue": "7b6de" } } }

Output -

Hope this is helpful. Please let me know incase of further queries.

Re: While configuring notebook session from datafactory pipeline, pipeline notebook activity fails

Anonymous — Fri, 15 Mar 2024 06:19:07 GMT

Hello Binit17 ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
Otherwise, will respond back with the more details and we will try to help .

Re: While configuring notebook session from datafactory pipeline, pipeline notebook activity fails

Anonymous — Mon, 18 Mar 2024 07:10:34 GMT

Hi Binit17 ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
Otherwise, will respond back with the more details and we will try to help .