Hello,
Last night, we started seeing widespread failures of notebooks running in pipelines which caused most of our nightly import jobs to fail. It looks like the errors are related to imports of internal dependencies. I tried running the notebooks using the 1.3 and 1.2 Spark runtimes. Here are a few examples of the errors that show up:
Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - ModuleNotFoundError, Error value - No module named 'fsspec_wrapper.trident.core'' :
Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - NameError, Error value - name 'mssparkutils' is not defined' :
The former looks like a sub-dependency of the pandas library. The latter is notebookutils which we haven't needed to explicitly import. In all cases, these look like bundled dependencies with another package or with the runtime.
This looks a lot like the outage from January 30th where the Fabric team had done some updates on the 1.3 runtime that caused dependency issues, but in that case we were able to workaround by downgrading to 1.2. In this case, both 1.3 and 1.2 have the same problem.
Notebooks with few or no dependencies invoked are running as expected.
Anyone else experiencing this?
