Git Branches - How to handle Notebook and Pipeline connections in different workspaces?

[tpenrose]

I have a DEV workspace: DEV1

Within this workspace are three items:

A lakehouse: LH1

A notebook: Notebook1

A pipeline: Pipeline1

The notebook points to LH1 as it's default lakehouse.

The pipeline has a single Lookup activity which points to LH1.

This workspace is connected to the Main branch of a git repo.

If I create a new branch ("feature1") and a new workspace ("DEV1-feature1"), Fabric will create the new branch and workspace and load all of the items listed above into the new "DEV1-feature1" workspace.

However, the notebook ("Notebook1") and the pipeline ("Pipeline1") in this new workspace will still reference the lakehouse ("LH1") from the orginial workspace ("DEV1").

I know how to repoint the notebook and the pipeline to the lakehouse within the new workspace.

However, if I do this and then merge my "feature1" branch back into the Main branch and then sync the original workspace ("DEV1") with the Main branch, then the DEV1 workspace's items will now be linking to the lakehouse in the "DEV1-feature1" workspace.

How can I ensure that notebooks and pipelines in a new branch reference a lakehouse within the "current" branch and not the original branch?

Without this ability, it appears that branching within Fabric is effectively useless. I'm really hoping I've missed something simple.

[tpenrose]

@storks Well, the Support ticket was closed back in July, but unfortunately there wasn't a solution.

Here are the exact words from Microsoft support:

Issue:

Git Branches - How to handle Notebook and Pipeline connections in different workspaces?

Solution:

It is by design that, when a new Workspace is established with a new branch from the Original Workspace via Git Integration, the Artifacts that are synchronized to this New Workspace will continue to refer to the Artifacts from the Original Workspace.

As previously mentioned, kindly utilize the following link to submit an Idea.

https://ideas.fabric.microsoft.com/

If the idea garners significant votes, the Product team will take it into consideration for future releases.

So, apparently this is all by design. Which if true, means that whoever designed this really needs to look for a new job.

Unfortunately, as of today, the branching capabilities, much like the deployment pipelines in Fabric are effectively broken and more or less unusable, except in very specific cases or very simple demos.

A real shame, and I hope that at some point in the future this will be resolved, but for now my team and I have had to abandon branching within Fabric, deployments take ages and general reliability of the Fabric platform has been patchy at best.

Needless to say, we are reconsidering our plans for future Fabric expansion.

[ptvinh]

Unfortunately, I have the same issue. My temporary solution is to add %%configure with default lakehouse at the top of notebook 😞  

[storks]

@tpenroseHas the support team found a solution for you and can you share it with us? looking forward to your response.

[tpenrose]

Thank you. I have now raised a Support ticket.

The Support ticket number is 2406280050002854.

[Anonymous]

Hi @tpenrose ,

Please reach out to our support team to gain deeper insights and explore potential solutions. It's highly recommended that you reach out to our support team. Their expertise will be invaluable in suggesting the most appropriate approach.

Please go ahead and raise a support ticket to reach our support team:

https://support.fabric.microsoft.com/support

After creating a Support ticket please provide the ticket number as it would help us to track for more information.