Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

60 Days of Data Days! Live and on-demand sessions, challenges, study groups and more! And it's all FREE!. Join now. Learn more

Reply
jnickell
Helper V
Helper V

Notebook parameter inaccessible in next code cell

I'm new to Data Engineering / Notebooks and am trying to follow a youtube video to setup a notebook for schema validation after data has been landed in Lakehouse files.  

 

I have a parameter cell with a 'fileToTest' and a output_table_name. When I try to use the 'fileToTest' parameter outside of the parameter code cell it doesn't work and I get a "NameError: name 'fileToTest' is not defined" error.

 

--- Updated with some additional findings

 

I only have 3 code cells.  

Cell 1: (Parameters)

 

 

fileToTest = "Files/folder/file.csv"
output_table_name = 'raw_users'

 

 

Cell 2  (attempting to install GreatExpectations

 

 

%pip install --q great_expectations

df = spark.read.format("csv").option("header","true").load(fileToTest)
display(df)

 

UPDATE:
IF I comment out the pip install command the spark.read operation will work. Not sure what this means. Of course in my notebook that means cell 3 fails when it tries to import great_expectations. 

 

Cell 3 (attempting to create validations within Great Expectations Context

 

 

import great_expectations as gx
gxContext = gx.get_context()
validator = gxContext.sources.pandas_default.read_csv(fileToTest)

 

 

 

Either of the references above to 'fileToTest' fail with the NameError.

If I move this code to Cell 1 it works w/o issue

 

 

df = spark.read.format("csv").option("header","true").load(fileToTest)
display(df)

 

 

 

For reference the original video is here:

https://youtu.be/wAayC-J9TsU?si=D25oMc7oZfpGFrxc

1 ACCEPTED SOLUTION
jnickell
Helper V
Helper V

Found the cause of my issue.  Learning newbie here.

spark.read works with the Files/.... path

great_Expectations / ?pandas?  requires /lakehouse/default/ to be prepended to the path

View solution in original post

2 REPLIES 2
frithjof_v
Community Champion
Community Champion

I don't have experience with parameter cell, however my initial thought when you got the "NameError: name 'fileToTest' is not defined" was that you had not executed (run) Cell 1 before you tried to use the fileToTest variable in another cell.

So therefore the 'fileToTest' variable didn't exist at that moment when you executed another cell.

jnickell
Helper V
Helper V

Found the cause of my issue.  Learning newbie here.

spark.read works with the Files/.... path

great_Expectations / ?pandas?  requires /lakehouse/default/ to be prepended to the path

Helpful resources

Announcements
FabCon and SQLCon Barcelona 2026

FabCon & SQLCon – Barcelona 2026

Join us in Barcelona for FabCon and SQLCon, the Fabric, Power BI, SQL, and AI community event. Save €200 with code FABCMTY200.

Fabric Community Sticker Design Challenge Barcelona Carousel

Fabric Community Sticker Challenge - Barcelona 2026

If you love stickers, then you will definitely want to check out our community sticker challenge, Barcelona edition!

60 days of Data Days Carousel

Data Days 2026

Join Data Days 2026: 60 days of free live/on-demand sessions, challenges, study groups, and certification opportunities.

June Fabric Update Carousel

Fabric Monthly Update - June 2026

Check out the June 2026 Fabric update to learn about new features.