Using Deployment Pipelines with Warehouses

[egons11]

Hi, there seems to be an issue, when I set up a deplyoment pipeline with Test and Production and assign Test and Production workspaces to it, the deployment pipeline stages are always out of sync when trying to deploy a warehouse.

I have a warehouse in the test workspaces and even if I delete the prod one and try to redeploy the test warehouse, it deploys succesfully but pipeline afterwards shows out of sync as "Different".

If I click review changes we can see that it always will be out of sync as it has "Z" next to it after all the timestamps, which is basically Zero timezone. This is very weird behavior as  I would expect it to be 1:1 after deleting it and redeploying.

[uselessai_in]

Hi, thanks for the question. Find all deployment related issues with their creative solutions here. I tested and implemented all of them. Hope it helps.

link - https://uselessai.in/dont-use-set-variable-activity-in-data-factory-pipeline-microsoft-fabric-bf74fa...

2. https://uselessai.in/microsoft-fabric-interesting-and-hidden-deployment-observations-important-bug-f...

3. https://uselessai.in/microsoft-fabric-stored-procedure-not-reflecting-warehouse-connection-change-in...

4. https://uselessai.in/microsoft-fabric-warehouse-deployment-issue-s-and-potential-solution-s-9ad36041...

[Anonymous]

Hello @egons11 ,

Thanks for using Fabric Community.
At this time, we are reaching out to the internal team to get some help on this .
We will update you once we hear back from them.

[Anonymous]

Hi @egons11 ,

Apologies for the delay in response.
We have an update from the internal team -

This is a known issue. ALM, diff for empty DWs show that there is a difference, in the xmla.json

This is only for empty DWs (the schema is empty). If the user will add any table, it will work correctly.

Since it is only for empty ones, it is not a high priority in the list of issues we have.

---

Hope this is helpful. Please let me know incase of further queries.

[Anonymous]

Hi @egons11 ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered.
Otherwise, will respond back with the more details and we will try to help .

[egons11]

I have a lakehouse in a test enviroment with a table and data in the table, but when I try to deploy it to the production workspace, the lakehouse redoply's but there is no tabe in it, only an empty lakehouse, even though the source lakehouse has the table. Why is this the case?

[Anonymous]

Hi @egons11 ,

Apologies for delay in response. Yes this is bu design, see the note here -
Lakehouse deployment pipelines and git integration - Microsoft Fabric | Microsoft Learn

However if you want this feature to be enable please provide your feedback.

Appreciate if you can provide feedback: Microsoft Idea  on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

I hope this information helps. Please do let us know if you have any further questions.

[egons11]

I see, very weird. How would one then go by having a test enviroment for a lakehouse and prod enviroment if any of the changes that would be done to tables and data would not be reflected after deployed to "prod" enviroment?

[Anonymous]

Hi @egons11 ,

We have a update from internal team -

It is by design. In delta world, where data can change on demand the schema, there is no easy solution.
Options are still being discussed but there is no plan to change that for now

---

Hope this is helpful.

[Anonymous]

Hi @egons11 ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered.
Otherwise, will respond back with the more details and we will try to help .

[egons11]

I understand the reply from the internal team, but it still doesn't answer -

"How would one then go by having a test enviroment for a lakehouse and prod enviroment if any of the changes that would be done to tables and data would not be reflected after deployed to "prod" enviromen"?

It would not be possible?

[Anonymous]

Hi @egons11 ,

Apologies for the inconvenience . However, At this moment, we might consider it to be a product limitation. For migrating tables and files , you might consider creating them manually in the target environment.
Thankyou.