Deployment pipeline anomalies in Fabric

[GammaRamma]

In Fabric, we used a deployment pipeline for the first time to deploy a dev workspace to a test workspace. This workspace contained a lakehouse, originally located in a folder within the workspace. After deployment, the semantic model and SQL analytics endpoint for the lakehouse appeared in the root folder of the test workspace, while the lakehouse itself was correctly placed in the designated folder. We tested this with multiple deployments and the bug happens every time. After moving the lakehouse to the root of the test workspace, we were able to move all three associated items back to the right folder, but this isn't ideal.

Additionally, the default semantic model and SQL analytics endpoint for the lakehouse and warehouse are now visible in the root folder of the test workspace. This visibility is different from the dev workspace, where these elements remained hidden. We seemingly cannot delete or move these items so this situation isn't ideal.

Here is a capture of the root of the test workspace after deployment. In contrast, the root of the dev workspace contains only the three original folders.

 Thank you for any help!

Update: We have since discovered several other bugs in our deployment pipeline.

First, the notebooks that were deployed from the dev workspace to the test workspace are still linked to the lakehouses in the dev workspace instead of being linked to the lakehouses that were copied to test from dev. Obviously, this is problematic, as changing all notebook linked lakehouses on every deployment is a significant amount of work and forgetting to do so will affect data integrity.

Second, when we look at the deployment pipeline itself, it says that several pipelines have differences between dev and test. When we look at the JSON code from the deployment pipeline, it is not the right JSON code for this pipeline. In fact, the pipelines are the same, as we can see when we look as the JSON code in the pipeline itself.

Third, when we try to re-deploy the entire workspace from dev to test, it is now raising an error and refusing to do so, saying that "Deployed items with the same name in the target and source stages are not bound". 

So it seems that deployment pipelines in Fabric are quite unstable and not ready to be actually used.

[avt]

@GammaRamma , I hit the exact same issue with folders and items being in the folder. Folders, as I understand are not supported items in deployment pipeline yet. however, once you move your lakehouse in test workspace from the folder to the root, like you mentioned, you can then move all 3 back into the folder. it is an inconvinience, however, it is still paired to the right source object. 
And regarding the notebook, have you tried deployement rules to make sure your default lakouse for notebooks in test WS is the test LH. this can be set after the initial deployment because you need to have the lakehouse first in the test and then subsequest deployments will use that rule and repoint your test notbooks to test lakehouse at the time of deployement. 
Hope this helps. Please do let me know if you ran into issues as we are developing exactly same thing that you are right now and it will  help us too

[GammaRamma]

@avt, thank you for your reply. When I deploy from dev to test and then go into the deployment rules of the test workspace, no other lakehouse is available to set as default for the my notebook. I have tried manually changing the default lakehouse in the test notebook and going to see in the deployment pipeline if the test lakehouse appeared but it doesn't. I also tried redeploying from dev and it put the dev lakehouse back where I had manually put the test lakehouse as default.

Furthermore, my notebook is linked to several lakehouses, so even if this feature did work, it still would not really have solved my problem, as it only sets the default lakehouse.

Thanks

[avt]

@GammaRamma , may I know where/how are you setting the default LH in your development WS notebook so I can help better?

[GammaRamma]

@avt, I am setting it from the notebook in the left side panel:

[avt]

@GammaRamma ,The default lakehouse in a notebook can be managed in the configuration settings of the notebook’s code. This should be set the beginning of your notebook.
%%configure

{

     "defaultLakehouse": {

             "name": "your-lakehouse-name", # The name of your lakehouse

             # "id": "<(optional) lakehouse-id>", # The ID of your lakehouse (optional)

             # "workspaceId": "<(optional) workspace-id-that-contains-the-lakehouse>"

             # The workspace ID if it's from another workspace (optional)

         }

 }
See if this helps setting up the deployment rules in the pipeline. 
Note: you can only set deployement rules after the inital deployment. in other words, you cannot set it befreo or for the first deployment. Also you have to be the owner of the notebook to set the deployment rules. 

Let me know if this helps.

[GammaRamma]

@avt, I executed the code in the notebook with the name of the lakehouse and the workspaceId. I am the owner of the notebook. I had deployed from dev to test yesterday. Now I open the deployment pipeline and the only option I see for the test notebook default lakehouse is the dev lakehouse still. The test lakehouse still does not appear. There are also some enigmatic N/A entries below the dev lakehouse: 

[avt]

Myapologies for missing this @GammaRamma , are you still facing the issue and need help or has it been solved?

[Joshrodgers123]

I hit the exact same issue as well and opened a support ticket today.

Deployment pipelines seem to be pretty broken with Fabric items.

[lbendlin]

Since you have a Pro license you can open a Pro ticket at https://admin.powerplatform.microsoft.com/newsupportticket/powerbi