Forum Discussion
Helper IIIEnvironment with Runtime 1.3 fails when uploading custom library
Helper IIIDowngrading to Runtime 1.2 does work with the same custom library.
But seems like a bad solution as the regular Workspace environment is Runtime 1.3 and we would be losing many improvements in our Data Flow by downgrading.
I have other Environment running Runtime 1.3 and the same library, but those were set up last month and those work fine.
Hi JosueMolina
If there is another environment running Runtime 1.3 and the same library working fine in the same workspace, you may set that environment as the default environment of the workspace in Worksapce Settings.
If there is no other environment working fine in the same workspace, you may still use Runtime 1.3 for the workspace environment, but create a custom environment running on Runtime 1.2 and upload the custom library. For some specific notebooks which uses the custom library, run them on the custom Runtime 1.2 environment. Run the other notebooks on the workspace default Runtime 1.3 environment. This may be a temporary alternative.
Feel free to let me know if you have any question.
Best Regards,
Jing
- JosueMolina
The working environment is in a different Workspace sadly.
Is there any issue that may come about for using Runtime 1.2 and Runtime 1.3 to write Delta Tables to the same Lakehouse in the same workspace? Default is 1.3 but some notebooks need this custom library and will be using 1.2 to write Delta Tables.Hi JosueMolina
Currently there is no general known issue related to using Runtime 1.2 and Runtime 1.3 to write Delta Tables to the same Lakehouse in the same workspace. But some key components in two Runtimes have different versions. Migrating from 1.3 to 1.2 may lead to some incompatibilities, if you've used some new features that are only supported in 1.3. It would be better to do some testing before the migration.
Apache Spark runtime in Fabric - Microsoft Fabric | Microsoft Learn
Best Regards,
Jing- JosueMolina
Agreed. I think our main failing was leveraging the 1.3 Runtime while still in preview.
We'll stick to 1.2 in the meantime.
Hi JosueMolina
If the working environment is in a different workspace, a possible alternative is that you can save a copy of the notebook into a different workspace, then run the copy notebook in that workspace to write data to the delta tables. In notebooks, it supports reading and writing data across workspaces by using the ABFS paths of the delta tables. For the copy notebooks, it can use environments of the new workspace.
Best Regards,
Jing