Forum Discussion

Dinosauris's avatar
Dinosauris
Helper I
2 years ago
Solved

Running a Fabric Notebook as Spark Job.

Hello,   I am interested in converting a Fabric notebook that I’ve created, containing multiple transformation steps, into a Spark job. However, I’m struggling to find comprehensive documentation o...
data engineering
  • Dinosauris's avatar
    Dinosauris
    2 years ago

    Hello!

     

    In the meantime we decided to run the Fabric notebooks via pipelines for the use case.

     

    Thank you anyways for the help! 🙂

HimanshuS-msft's avatar
HimanshuS-msft
Microsoft Employee
2 years ago

Hello Dinosauris  , 

Just to provide a bit more clarity here . Since the transformation is already there in the notebbok , you can add that to a pipeline ( as Anonymous called out above ) or you can scheduled from the notebook pane itself . The below snapshot should help . 

 

Thanks 
Himanshu

  • Dinosauris's avatar
    Dinosauris
    Helper I
    2 years ago

    Hello Himanshu,

     

    thanks for the info. I am aware of this and this is my backup option (and will probably be the solution of the problem). However, I still wanted to understand why I cannot make the Notebook work using  Spark Job. I have a lot of display() functions in there and don't know whether this could be an issue. No mssparkutils APIs are called.

     

    Thanks & BR 🙂