topic Re: Microsoft fabric WareHouse migration issue via deployment Pipeline in Data Engineering

Microsoft fabric WareHouse migration issue via deployment Pipeline

Umasankar_Sane — Wed, 13 Nov 2024 14:52:54 GMT

Getting this common error many times and does not give any clue how to resolve this issue

Re: Microsoft fabric WareHouse migration issue via deployment Pipeline

Anonymous — Thu, 14 Nov 2024 06:17:04 GMT

Does this warehouse reference a lakehouse? If so, this might be a cause of the error. As the Lakehouses are not supported enough be deployment pipelines. To resolve this, you may try two options:

Manually create the lakehouse in the destination workspace WITH all the tables and views that exist in the source workspace.
Remove any reference to the lakehouse in the warehouse. (Note: Second option is only needed if for some reason first option is not viable.)

If above options don't work, and the warehouse doesn't reference a lakehouse, please do following steps to try to find more detailed error messages:

1) Export source datawarehouse - click the "Download SQL database project" from within the warehouse editor:

2) Create a new warehouse

3) Import the zip file exported in step 1 INTO DESTINATION FOLDER (this is because the warehouse may be referencing a table in a different artifact that does exist in the source but not the destination):

4) If import fails, it should show the details of the failure. Based on this, you can determine what needs to be fixed for the pipeline to succeed.

If the import is successful, but the deployment pipeline isn't, please open a support ticket to Microsoft to investigate this issue further.

Best Regards,
Jing
If this post helps, please Accept it as Solution to help other members find it. Appreciate your Kudos!

Re: Microsoft fabric WareHouse migration issue via deployment Pipeline

Umasankar_Sane — Thu, 14 Nov 2024 08:28:12 GMT

Hi Jing , Thank you!
Yes, we have references to Lake house. But in destination workspace we have all lake house tables created first beforing trying with migration
Let me try to import as you suggested in Second option and post the observations here .

Update :

I have tried import is successful , but not able to migrate to different ws as part of deployment

Re: Microsoft fabric WareHouse migration issue via deployment Pipeline

Anonymous — Fri, 15 Nov 2024 06:06:44 GMT

Hi Umasankar_Sane

I found this issue has been submitted internally. Engineers are looking for solutions for it but there isn't ETA available. I will keep tracking that and keep you updated of any progress. Thanks for your patience.

If your case is urgent, please open a support ticket to Microsoft. Engineers will contact you and investigate it further. They are able to check some backend logs for troubleshooting.

In addition, have you tried this option?

-- Remove any reference to the lakehouse in the warehouse. (Note: Second option is only needed if for some reason first option is not viable.)

This option may require you to remove the references manually, deploy the warehouse, then recreate the references in the destination workspace manually. I understand this is not a workaround. But if it could work, it can help you move forward until a solid solution or fix is available.

Best Regards,
Jing

Re: Microsoft fabric WareHouse migration issue via deployment Pipeline

hrosillo — Wed, 26 Feb 2025 20:44:26 GMT

Very good discussion. My question is, when I publish it with "SQL Database Project" in the construction and publishing of schemas, it validates building objects, publishing schemas, importing user queries, and wrapping things up.

Does this show errors of invalid references?

Broken foreign keys (Foreign Key Violations)
Orphan records
Broken data dependencies
Missing or inconsistent data

I ask this because there is a question related to this discussion.

You have a Fabric workspace named Workspace1 that contains a warehouse named Warehouse1.
You plan to deploy Warehouse1 to a new workspace named Workspace2.
As part of the deployment process, you need to verify whether Warehouse1 contains invalid references. The solution must minimize development effort.
What should you use?

A. a database project
B. a deployment pipeline
C. a Python script
D. a T-SQL script