Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago
Solved

mssparkutils run notebook across workspaces

Greetings! I am trying to use the mssparkutils.notebook.run() function to run a notebook in a different workspace. I get the same error for both these variants:     mssparkutils.notebook.run(pat...
data engineering
  • Anonymous's avatar
    Anonymous
    2 years ago

    Well, I kept the Spark Version low, because otherwise I had minReaderVersion compatibility issues. I will try it, but this minReaderVersion stuff broke pretty much everything.

     

    Edit: Nope, still doesn't work, unfortunately, will look for a fix, but the issue in this thread is the minor one compared to the minReaderVersion.

    Py4JJavaError: An error occurred while calling o4423.throwExceptionIfHave.
: com.microsoft.spark.notebook.msutils.NotebookExecutionException: Error while decoding: java.lang.IllegalArgumentException: requirement failed: Mismatched minReaderVersion and readerFeatures.
Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi Anonymous 
You are getting this error because you are using Spark Runtime1.1 (spark3.3) in the Notebook. Please change to Runtime 1.2 and run the code.


Hope this helps. Please let me know if the issue still persists.

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Well, I kept the Spark Version low, because otherwise I had minReaderVersion compatibility issues. I will try it, but this minReaderVersion stuff broke pretty much everything.

 

Edit: Nope, still doesn't work, unfortunately, will look for a fix, but the issue in this thread is the minor one compared to the minReaderVersion.

Py4JJavaError: An error occurred while calling o4423.throwExceptionIfHave.
: com.microsoft.spark.notebook.msutils.NotebookExecutionException: Error while decoding: java.lang.IllegalArgumentException: requirement failed: Mismatched minReaderVersion and readerFeatures.