Forum Discussion
jisaenz
Helper I
Helper IPerformance issues running pipelines and notebooks
I am working with the trial capacity and I am experiencing performance issues excuting pipelines and notebooks that use spark. They are running too slow or never finish. I would like to know what is...
arjaano
Advocate II
Advocate III'm experiencing the same issue.
Running a small notebook by hand takes about 1 minute. The runtime for running the exact same notebook from a pipeline is over 5 minutes. The notebook is ingesting 10 small CSV-files into lakehouse tables, nothing fancy.