Forum Discussion

ToddChitt's avatar
ToddChitt
Super User
2 years ago
Solved

Notebook fails with error 200 when run from Pipeline

I'm fairly new to Spark Notebooks. I have one that dumps a JSON file into a couple of tables in my Lakehouse. It works fine when the Notebook is run on its own. But when I try to run it from the cont...
data engineering
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi ToddChitt 

    Thanks for the details. Can you please send me the code which you are running in the notebook? I will try to repro this from my side and let you know.

    And also can you please try to create a new notebook and then attach the lakehouse. Now try to run this notebook from the pipeline. If the issue still persists please do let me know. This way there wont be a grey out option and you can attach or detach the lakehouse.


    Did you try to set the target lakehouse as the default lakehouse? Default lakehouse is identified by the pin icon.

     



    Thanks.

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi ToddChitt 

Thanks for the code. Glad that your query got resolved. Please continue using Fabric Community for any help regarding your queries.

Anonymous's avatar
Anonymous
Not applicable
2 years ago

hello v-nikhilan-msft,

I am getting similar error but it is about variable/dataframe note defined.. Name Error. When I ran notebook manually, it is successfull but when triggered from pipeline it is failing with exceptions mentioned above.

 

below is the line of code causing error, where read_particular_day_files is a user defined function.

numfiles_pti_ar, df_toprocess_pti_ar, df_policy_transaction_info_ar = read_particular_day_files(accountingDate, '', 'PolicyTransactionInfo')
 
please help me resolve this issue
  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi Anonymous 

    Thanks for the post. But as the initial ask is different from your issue, can you please create a new post and tag me. I will surely help.

    Please attach some screenshots of the error too.

    Thanks.