Forum Discussion
Managing Spark Pools (Confusing, no UI, no souce control)
Where do we go to edit/manage the pools in a workspace?
The only place I can find is the funky little dropdown widget that is used for setting the default pool:
... if you open the dropdown on the "default pool" then there is a hidden option in there to create a new pool.
The list of spark pools seem fairly important, and shouldn't be hidden within the dropdown menu of the "default pool". Where do I view and manage the list of pools? Am I missing another screen? Also, how do I extract this information to my source control?
Sorry for the basic questions. Sometimes these U/I's in Fabric are supposed to make the environment "easy" but they end up having the opposite effect.
4 Replies
So you are saying that the ONLY place to maintain Spark pools is hidden, and can only be found within a dropdown widget? (... one that selects a "default pool" for Spark jobs).
That is insanity.
Saying it is "not the best UI" is an understatement. I think any random highschooler using winforms would build a better UI than this. I suspect that Microsoft isn't counting for serious Spark workloads to run on this platform. (I think they make a good profit on Azure Databricks as well, and probably don't put all their eggs in one basket)
Hi dbeavon3 ,
Can you please clarify your question? Are you trying to look for a UI that lists in a tabular fashion of all the pools and their configuration settings (Starter plus any that you created) in the Workspace? Unfortunately there isn't a screen like that yet, but perhaps try using PowerShell or the Fabric REST API to extract the details for all of them at once.
Exactly. If a "custom pool" is analogous to a cluster, there should be a maintenance screen for them. I've never worked with a platform where a cluster entity was so hard to find in the UI.
I like the idea of the powershell/local management. Hopefuly Microsoft will take that one step further and build a thick Visual Studio management interface for our Fabric workspaces. The UI that they built in their Power BI portal is really not very good.
