topic Need to pass multiple values as parameters in notebook by using pyspark in Pipelines

Need to pass multiple values as parameters in notebook by using pyspark

sudhav — Fri, 29 Sep 2023 15:17:34 GMT

Hi Team,

I have a notebook using pyspark.

there i have written a code that is fetching share market value for every 5 mins. below is my notebook code

In above snap, i have given company name as ABC and i have parameterized that cell and i am getting below results in my lakehouse as table and adding a single row for ABC company for evry 5 mins(i scheduled it for 5 mins)

now my question is i need to get some more companies data like ABC,XYZ,PQR companies data in that same table as rows one after other as rows, it means for every 5 mins 3 rows should add(one row for each company)

I was able to get this using pipeline using config table, but i need to achieve it by using only notebook and giving parameters list in the notebook.

can somebody look into it and do needful.

TIA

Re: Need to pass multiple values as parameters in notebook by using pyspark

ajarora — Fri, 29 Sep 2023 19:00:32 GMT

Did you check this : https://learn.microsoft.com/en-us/fabric/data-factory/notebook-activity ?

Re: Need to pass multiple values as parameters in notebook by using pyspark

Anonymous — Mon, 02 Oct 2023 10:03:11 GMT

Hi sudhav ,
Thanks for using Fabric Community.
Could you please send the screenshot of the notebook code which you are using for loading the data into Lakehouse table?
This would help us in understanding the ask better.

Re: Need to pass multiple values as parameters in notebook by using pyspark

Anonymous — Thu, 05 Oct 2023 06:51:17 GMT

Hi sudhav ,

We haven’t heard from you on the last response and was just checking back to see if you could provide the details asked above. Please let us know if you have any further queries

Re: Need to pass multiple values as parameters in notebook by using pyspark

Anonymous — Fri, 06 Oct 2023 09:22:28 GMT

Hi sudhav ,
Following up to check whether your issue has been resolved. Please do let us know if you have any further queries.

Re: Need to pass multiple values as parameters in notebook by using pyspark

12angrymentiger — Wed, 06 Dec 2023 21:39:33 GMT

Here's a really simple example.

In your parent notebook you would set a dictionary from a list of keys/values, then you would call the child notebook (Notebook 1) in this case by passing in that dictionary as a parameter. You could then do anything you want with the values coming in.

Parent Notebook:

Notebook 1:

Make sure to set the companylist as a parameter

Results: