Forum Discussion
Environment for Python packages to be used by different PySpark scripts in different Workspaces
this still has not solved my issue. Everything I need those identical libraries to be used in other workspaces, I need to create another environment in the workspace, and the creation of new environment takes a long time. Can Microsoft looks into how to have a better solution to this?
Hi tan_thiamhuat,
No, environments are specific to the workspace they are in, you can't share an environment between workspaces.
Create, Configure, and Use an Environment in Fabric - Microsoft Fabric | Microsoft Learn
If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
yes, I understand that.. but I tested out. At time, it works across different workspaces. That is why I get confused.
Hi tan_thiamhuat,
Sorry, I stand corrected. Seems the MVPs from the Reactor session today were also incorrect:
Create, Configure, and Use an Environment in Fabric - Microsoft Fabric | Microsoft Learn
The environment is available on both the Notebook and Spark Job Definition tabs. When notebooks and Spark job definitions are attached to an environment, they can access its libraries, compute configurations, and resources. The Explorer lists all available environments that are shared with you, are from the current workspace, and are from other workspaces to which you have access.
When you attach an environment from another workspace, both workspaces must have the same capacity and network security settings. Although you can select environments from workspaces with different capacities or network security settings, the session fails to start.
When you attach an environment from another workspace, the compute configuration in that environment is ignored. Instead, the pool and compute configurations default to the settings of your current workspace.It seems this is supported.
Are your workspaces on the same capacity and have the same network security settings?
If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
I am using the free tier (I suppose no capacity, so it is the same capacity), and same network security settings. How can we create an environment outside of any workspace, so that whichever PySpark notebook in whichever workspace can access that environment without any issue? Else users would need to duplicate the same environment in each workspace if their PySpark notebooks in different workspaces have identical needs for that environment. Not sure if my requirements are clear.
this still has not solved my issue. Everything I need those identical libraries to be used in other workspaces, I need to create another environment in the workspace, and the creation of new environment takes a long time. Can Microsoft looks into how to have a better solution to this?