topic Re: Git branch - using spark environment deletes public and custom packages in Data Science

Git branch - using spark environment deletes public and custom packages

KimTutein — Fri, 17 Jan 2025 09:53:17 GMT

Hi Community

Does anyone have experiance with branching off in git and using environment in spark. The usecase I have is I have a workspace with our code items (pipelines, notebooks and environments). The workspace is connected to git (DevOps). I am able to branch of main and make for instance feature_1 branch and synch with new workspace. Every item is recreated in new workspace but for environment I see two problems

All public and custom library is deleted from the environment
We will need to publish environment again from new workspace which will take quite some time (+15 minutes)

I would love just to reuse environment from original branch but it does not seem possible.

Anyone who will share experience with branching off and using environments?

Friendly Regards

Kim Tutein

Re: Git branch - using spark environment deletes public and custom packages

Anonymous — Tue, 21 Jan 2025 10:02:04 GMT

Hi KimTutein ,

Thank you for reaching out to the Microsoft Fabric Community Forum about the issue you are encountering.

Providing the resolution steps which might assist you in resolving the issue:

Before branching off,export libraries from the old workspace using REST API.
After creating new workspace, reinstall libraries in the new environment.
For the second one,Create base environment templates to quickly set up the same environment for new workspaces.

If still facing difficulty in resolving the issue, please feel free to reachout.

If this post helps, please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Re: Git branch - using spark environment deletes public and custom packages

KimTutein — Tue, 21 Jan 2025 13:30:38 GMT

Hi Anonymous

Thank you for reaching out. I get we could do the above which is indeed what I have been doing (by hand though and not REST API). However I feel that it is a bug that one cannot make a new wokrspace using the gui and correctly synch from git into that (doing this by hand add extra work and is an operational risk - I would imagine that one develober foregot this and merge back into main - the develober could have been changing just a data pipeline). I have made a incident to the support and they are looking further into it.

Thank you very much for your input - I have accepted as a Solution

Regards

Kim