topic Re: Call one notebook from another with parameters in Data Engineering

Call one notebook from another with parameters

VickyDev18 — Fri, 16 Feb 2024 15:45:34 GMT

I have this simple notebook to load data from an xlsx file in a lakehouse to a delta table in the Lakehouse.

How can I make this reusable/parameterized so I call it from another notebook and pass parameters and how exactly would I call it from another notebook?

import pandas as pd source_path = "/lakehouse/default/" source_file_name = "Files/Test.xlsx" sheet_name = "Test" destination_table = "test" # Read Excel data into a Pandas DataFrame df = pd.read_excel(source_path + source_file_name, sheet_name=sheet_name) # Convert Pandas DataFrame to Spark DataFrame df_spark = spark.createDataFrame(df) # Save Spark DataFrame as a Delta table df_spark.write.mode("overwrite").format("delta").saveAsTable(destination_table)

I am aware of how to call a parameterized notebook from a pipeline How to Pass Parameters from Pipelines to Notebooks in Microsoft Fabric! - YouTube.

However, I want to know if we can call the notebook from another notebook.

Re: Call one notebook from another with parameters

HimanshuS-msft — Fri, 16 Feb 2024 20:10:43 GMT

Hello @VickyDev18 ,

Its me again. Have you tried using this

from notebookutils import mssparkutils

mssparkutils.notebook.runMultiple(["aaa"])

Re: Call one notebook from another with parameters

Anonymous — Mon, 19 Feb 2024 05:11:17 GMT

Hi @VickyDev18
Thanks for using Fabric Community.
You can call one notebook from another notebook using %run command. I have created a repro of the same but instead I used a CSV file. I have attached the screenshots of the code.

Functions is my child notebook where I am defining a function to create a table inside lakehouse using the data inside a CSV file.

I am calling this notebook from another notebook:

The table got created in the lakehouse.

Hope this helps. Please let me know if you have any further questions.

Re: Call one notebook from another with parameters

Anonymous — Tue, 20 Feb 2024 10:43:32 GMT

Hi @VickyDev18
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. Otherwise, will respond back with the more details and we will try to help.
Thanks

Re: Call one notebook from another with parameters

VickyDev18 — Tue, 20 Feb 2024 12:38:24 GMT

I had tried this method but for some reason it wasn't working. However, now that I tried it again, it worked!

Only simplified approach I tried was to just call the function directly without the 2nd function.

So overall the approach was:-

1. Create a Notebook called Functions with code below.

import pandas as pd def load_excel(source_path, source_file_name, sheet_name, destination_table): # Read Excel data into a Pandas DataFrame df = pd.read_excel(source_path + source_file_name, sheet_name=sheet_name) # Convert Pandas DataFrame to Spark DataFrame df_spark = spark.createDataFrame(df) # Save Spark DataFrame as a Delta table df_spark.write.mode("overwrite").format("delta").saveAsTable(destination_table)

2. Create a new Notebook and call the function above using the code below

%run Functions #load_excel(source_path, source_file_name, sheet_name, destination_table) load_excel("/lakehouse/default/Files/", "Test.xlsx", "Test", "test")

Re: Call one notebook from another with parameters

Anonymous — Tue, 20 Feb 2024 13:54:40 GMT

Hi @VickyDev18
Thanks for sharing your approach here. Glad that your query got resolved. Please continue using Fabric Community for any help regarding your queries.