topic Re: Python Notebook does not respect pipeline input parameter in Data Engineering

Python Notebook does not respect pipeline input parameter

Anonymous — Thu, 17 Jul 2025 22:20:41 GMT

Hi,

I'm encountering an issue when running a Python notebook through a Fabric pipeline. When I run the notebook manually in the workspace, everything works fine: it correctly loads the Excel file from SharePoint using the sheet name I provide.

However, when I run the same notebook via a pipeline, the execution fails with this error:

ValueError: Worksheet named 'Sheet1' not found

This shows that the sheet parameter is not correctly passed to the notebook inside the pipeline context — it seems to default to "Sheet1", even though "Sheet2" is clearly defined in the pipeline input parameters (see attached screenshots).

Expected behavior:
The notebook in the pipeline should honor the runtime parameter and use "Sheet2" as provided, instead of defaulting to "Sheet1".

Environment:
- Microsoft Fabric (Trial version)
- Python 3.11 notebook
- Excel file hosted on SharePoint

Re: Python Notebook does not respect pipeline input parameter

AmiGarala — Fri, 18 Jul 2025 05:33:30 GMT

1. Check Notebook Parameter Cell Format
Your notebook must have a parameter cell defined like this at the top:

# Parameters
sheet_name = "Sheet1" # default value
Fabric uses this cell to inject pipeline parameters. If this is missing or misnamed, the pipeline cannot override it.

Make sure:

The variable name (sheet_name) matches exactly with the name used in the pipeline.

The cell is tagged with "parameters" (Fabric auto-detects this if you're using the right format).

2. Ensure the Pipeline Is Passing the Parameter
In your pipeline:

Go to the notebook activity block.

Click on the "Settings" pane.

Confirm:

Input parameter name is sheet_name (case-sensitive match).

Value is set to Sheet2.

Optional sanity check: Add a debug cell to your notebook to log it:
print(f"Using sheet name: {sheet_name}")

3. Double-Check Parameter Mapping in JSON Behind the Scenes (Optional)
If you're comfortable editing JSON:

Open the pipeline JSON (if accessible via export or dev mode).

Look for something like:

"notebookActivity": {
"parameters": {
"sheet_name": "Sheet2"
}
}

4. Avoid Hardcoded Values in Notebook
Sometimes developers accidentally override the parameter later in the notebook:

sheet_name = "Sheet1" # This will override pipeline input
Make sure this line appears only in the parameter cell and not anywhere else.

Re: Python Notebook does not respect pipeline input parameter

Anonymous — Fri, 18 Jul 2025 07:38:30 GMT

Hi @AmiGarala and thank you for your quick response.
Placing the configured cell at the very top fixed my issue, there was also a misnamed parameter. Thanks again.

Re: Python Notebook does not respect pipeline input parameter

BhaveshPatel — Fri, 18 Jul 2025 07:50:48 GMT

For that, You should use Dataflow Gen 2 rather than Fabric Data Factory.

Here is a quick view

Easy to implement and all is available:

Re: Python Notebook does not respect pipeline input parameter

v-agajavelly — Mon, 21 Jul 2025 11:30:44 GMT

Hi @Anonymous ,

Thank you @AmiGarala Your step-by-step breakdown was spot-on. Highlighting the need for the correctly named parameter cell at the top and ensuring no overriding later in the notebook really clarified the root cause. Appreciate the depth and clarity of your solution.

@Anonymous Glad to hear the issue is now resolved! Out of curiosity, do you see potential for applying this pattern more broadly in your notebooks to help future-proof parameter handling in pipelines?

If this response helps, consider marking it as “Accept as solution” and giving a “kudos” to assist other community members.

Thank you,
Akhil.

Re: Python Notebook does not respect pipeline input parameter

v-agajavelly — Wed, 23 Jul 2025 08:56:35 GMT

Hey @Anonymous ,

Just wanted to follow up and see as super users suggested if you’ve had a chance to explore using that parameter pattern in any of your other notebooks? It could be a nice little win for making things more pipeline-friendly down the line curious to hear if you're thinking about applying it more broadly.

Regards,
Akhil.