Forum Discussion
Fabric Capacity and Data Engineering Development
Hello nnshr
Yes, you can absolutely follow an approach where developers develop reusable Python code locally and make that code available on Fabric. To do that you use an Environment Fabric item. You can upload custom Python libraries in a Environment item and set that item as a default environment in your Spark settings.
The Spark settings can be found in Workspace Settings > Data Engineering/Science.
Hope this helps - please appreciate by leaving a Kudos or accepting as a Solution!
Hi nnshr ,
Yes, what you’re experiencing is quite common in Fabric environments with multiple engineers.
Spark notebooks and pipelines consume capacity at the workspace level, and even idle or long-running Spark sessions can quickly block other users when capacity is limited.
A good pattern is to separate concerns:
- Develop reusable business logic as Python packages locally, with proper testing and CI
- Keep Fabric notebooks thin, focusing mainly on orchestration and execution
This approach won’t remove Fabric capacity usage entirely, but it does reduce iteration cost and contention during development.
At the moment, Fabric notebooks themselves can’t be fully developed or executed locally, so capacity planning, session management, and controlling parallel execution remain key.