Forum Discussion

nnshr's avatar
nnshr
Frequent Visitor
6 months ago
Solved

Fabric Capacity and Data Engineering Development

I'm currently working in an organization that is using Fabric for data engineering, we are utilizing pipelines and notebooks. We have a team of about 6 data engineers. We are encountering many diffi...
Data Engineering
Data Pipeline
Notebook
  • deborshi_nag's avatar
    deborshi_nag
    6 months ago

    Hello nnshr 

     

    Yes, you can absolutely follow an approach where developers develop reusable Python code locally and make that code available on Fabric. To do that you use an Environment Fabric item. You can upload custom Python libraries in a Environment item and set that item as a default environment in your Spark settings. 

     

    The Spark settings can be found in Workspace Settings > Data Engineering/Science. 

     

    Hope this helps - please appreciate by leaving a Kudos or accepting as a Solution! 

deborshi_nag's avatar
deborshi_nag
Super User
6 months ago

Hello nnshr 

 

Yes, you can absolutely follow an approach where developers develop reusable Python code locally and make that code available on Fabric. To do that you use an Environment Fabric item. You can upload custom Python libraries in a Environment item and set that item as a default environment in your Spark settings. 

 

The Spark settings can be found in Workspace Settings > Data Engineering/Science. 

 

Hope this helps - please appreciate by leaving a Kudos or accepting as a Solution!