Forum Discussion

JosueMolina's avatar
JosueMolina
Icon for Helper III rankHelper III
2 years ago

Environment with Runtime 1.3 fails when uploading custom library

I began experiencing issues when loading a custom library into our Environment yesterday.    I have other environments in different workspaces with the same library working fine.  I found this e...
data engineering
environment
Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi JosueMolina 

 

It is possible that not all environments where custom libraries are installed will get an error when running a notebook. However, it cannot be ruled out that the failure to install custom libraries in a new environment has nothing to do with this known issue.

 

From the error message you got, it seems it failed to parse dependencies of notebookutils. While notebookutils is a built-in package in the Runtime. So it seems a Runtime issue. 

 

Have you finished testing downgrading this environment to Fabric Runtime 1.2? Did it work well with Fabric Runtime 1.2?

 

Best Regards,
Jing

JosueMolina's avatar
JosueMolina
Icon for Helper III rankHelper III
2 years ago

Downgrading to Runtime 1.2 does work with the same custom library.

But seems like a bad solution as the regular Workspace environment is Runtime 1.3 and we would be losing many improvements in our Data Flow by downgrading.

I have other Environment running Runtime 1.3 and the same library, but those were set up last month and those work fine.

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi JosueMolina 

     

    If there is another environment running Runtime 1.3 and the same library working fine in the same workspace, you may set that environment as the default environment of the workspace in Worksapce Settings. 

    If there is no other environment working fine in the same workspace, you may still use Runtime 1.3 for the workspace environment, but create a custom environment running on Runtime 1.2 and upload the custom library. For some specific notebooks which uses the custom library, run them on the custom Runtime 1.2 environment. Run the other notebooks on the workspace default Runtime 1.3 environment. This may be a temporary alternative. 

     

    Feel free to let me know if you have any question. 

     

    Best Regards,
    Jing

    • JosueMolina's avatar
      JosueMolina
      Icon for Helper III rankHelper III
      2 years ago

      The working environment is in a different Workspace sadly.

      Is there any issue that may come about for using Runtime 1.2 and Runtime 1.3 to write Delta Tables to the same Lakehouse in the same workspace? Default is 1.3 but some notebooks need this custom library and will be using 1.2 to write Delta Tables.

      • Anonymous's avatar
        Anonymous
        Not applicable
        2 years ago

        Hi JosueMolina 

         

        Currently there is no general known issue related to using Runtime 1.2 and Runtime 1.3 to write Delta Tables to the same Lakehouse in the same workspace. But some key components in two Runtimes have different versions. Migrating from 1.3 to 1.2 may lead to some incompatibilities, if you've used some new features that are only supported in 1.3. It would be better to do some testing before the migration. 

        Apache Spark runtime in Fabric - Microsoft Fabric | Microsoft Learn

         

        Best Regards,
        Jing