Forum Discussion
Environment for Python packages to be used by different PySpark scripts in different Workspaces
this still has not solved my issue. Everything I need those identical libraries to be used in other workspaces, I need to create another environment in the workspace, and the creation of new environment takes a long time. Can Microsoft looks into how to have a better solution to this?
yes, I understand that.. but I tested out. At time, it works across different workspaces. That is why I get confused.
Hi tan_thiamhuat,
Sorry, I stand corrected. Seems the MVPs from the Reactor session today were also incorrect:
Create, Configure, and Use an Environment in Fabric - Microsoft Fabric | Microsoft Learn
The environment is available on both the Notebook and Spark Job Definition tabs. When notebooks and Spark job definitions are attached to an environment, they can access its libraries, compute configurations, and resources. The Explorer lists all available environments that are shared with you, are from the current workspace, and are from other workspaces to which you have access.
When you attach an environment from another workspace, both workspaces must have the same capacity and network security settings. Although you can select environments from workspaces with different capacities or network security settings, the session fails to start.
When you attach an environment from another workspace, the compute configuration in that environment is ignored. Instead, the pool and compute configurations default to the settings of your current workspace.
It seems this is supported.
Are your workspaces on the same capacity and have the same network security settings?
If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
I am using the free tier (I suppose no capacity, so it is the same capacity), and same network security settings. How can we create an environment outside of any workspace, so that whichever PySpark notebook in whichever workspace can access that environment without any issue? Else users would need to duplicate the same environment in each workspace if their PySpark notebooks in different workspaces have identical needs for that environment. Not sure if my requirements are clear.
Hi tan_thiamhuat,
Do you mean you're on a free trial capacity? There is no free tier of Fabric, and to be able to create fabric items like notebooks and environments, the workspace needs to be attached to a fabric capcaity.
Fabric items can't exist outside of workspaces, so you can't make an environment outside of any workspace.
If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
there is a Trial period at the top of my Fabric.