Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Grow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.

Reply
pshepunde
Helper I
Helper I

Notebook fails with error 200 when run from Pipeline but manual run is successful

Every time, it is failing with error like variable/dataframe not defined, when triggered from pipeline. But when I executed notebook manually, it ran without error. Please help me resolve this issue. Same pipeline ran successfully before.

 

Operation on target Bronze Layer failed: Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - Py4JJavaError, Error value - An error occurred while calling o4944.throwExceptionIfHave.
: com.microsoft.spark.notebook.msutils.NotebookExecutionException: name 'numfiles_pti_aric' is not defined
---------------------------------------------------------------------------NameError  

1 ACCEPTED SOLUTION

Hi @pshepunde 
Thanks for providing the details.
The error message you’re seeing is a NameError in Python, which means that the variable numfiles_pti_aric is not defined before it’s being used in the if statement. The line of code you’ve shown is calling a function read_particular_day_files which is supposed to return three values, one of which is numfiles_pti_aric. If this function call is executed before the if statement where numfiles_pti_aric is used, then this error should not occur.
Please ensure that the function read_particular_day_files is correctly defined and is indeed returning the expected values. Also, make sure that the function call is executed before the if statement.

Possible Solutions:

  1. Check Function Call: Ensure that the function read_particular_day_files is called before the if statement and that it's assigning the return values correctly. The function should be returning three values, and you should unpack them into separate variables.

  2. Define Variable Before Use: If numfiles_pti_aric is not supposed to come from the function call, you need to define it with an initial value before using it in the if statement. Just create the variable initially as numfiles_pti_aric=0 in the beginning of the code.

Hope this helps. Please let me know if you have any further questions.

 

View solution in original post

7 REPLIES 7
pshepunde
Helper I
Helper I

pshepunde_0-1715092881388.png

Hello @v-nikhilan-msft , please find the screenshot for more details. Thanks in advance

error message as below:

Operation on target Bronze Layer failed: Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - Py4JJavaError, Error value - An error occurred while calling o4944.throwExceptionIfHave.
: com.microsoft.spark.notebook.msutils.NotebookExecutionException: name 'numfiles_pti_aric' is not defined
---------------------------------------------------------------------------NameError                                 Traceback (most recent call last)Cell In[37], line 2
      1 from notebookutils import mssparkutils
----> 2 if numfiles_pti_aric==0:
      3     print(f"No Files exists for PolicyTransactionInfoARIC for {accountingDate}")
      4     mssparkutils.notebook.exit("NoFileExists_ARIC")
NameError: name 'numfiles_pti_aric' is not definedYou can check driver log or snapshot for detailed error info!  

 

line of code as below: It is a function call which returns 2 dataframes and one numeric value

numfiles_pti_aric, df_toprocess_pti_aric, df_policy_transaction_info_aric = read_particular_day_files(accountingDate, '', 'PolicyTransactionInfo')
 

Hi @pshepunde 
Thanks for providing the details.
The error message you’re seeing is a NameError in Python, which means that the variable numfiles_pti_aric is not defined before it’s being used in the if statement. The line of code you’ve shown is calling a function read_particular_day_files which is supposed to return three values, one of which is numfiles_pti_aric. If this function call is executed before the if statement where numfiles_pti_aric is used, then this error should not occur.
Please ensure that the function read_particular_day_files is correctly defined and is indeed returning the expected values. Also, make sure that the function call is executed before the if statement.

Possible Solutions:

  1. Check Function Call: Ensure that the function read_particular_day_files is called before the if statement and that it's assigning the return values correctly. The function should be returning three values, and you should unpack them into separate variables.

  2. Define Variable Before Use: If numfiles_pti_aric is not supposed to come from the function call, you need to define it with an initial value before using it in the if statement. Just create the variable initially as numfiles_pti_aric=0 in the beginning of the code.

Hope this helps. Please let me know if you have any further questions.

 

Ideally, we don't have to declare varibales or dataframes before assignment, but in this case, I declared variables and dataframes and then use them in function call. It worked for now, Thanks a lot!

Hi @pshepunde 
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. Otherwise, will respond back with the more details and we will try to help.
Thanks

v-nikhilan-msft
Community Support
Community Support

Hi @pshepunde 

Thanks for using Fabric Community.

Can you please provide the screenshot of the code along with the error message? This would help me to understand the issue better.

 

Thanks

Thanks for the details @pshepunde 

Can you please send me the screenshot of the code used in the notebook?

Thanks

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

Expanding the Synapse Forums

New forum boards available in Synapse

Ask questions in Data Engineering, Data Science, Data Warehouse and General Discussion.

RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

MayFBCUpdateCarousel

Fabric Monthly Update - May 2024

Check out the May 2024 Fabric update to learn about new features.