Pipeline deployment failing - deploying a Lakehouse from a dev to Test

[Naman_]

I am facing an issue while deploying a Lakehouse from a development workspace to a test workspace using Microsoft Fabric deployment pipelines. The deployment process fails with the following error details:

• Operation ID: cba095bf-a58c-41a7-ac0a-a31e7e90d275
• Correlation ID: 14d6372e-536d-7f14-eed9-3961f4062f2d
• Request ID: 04644dc3-2729-5846-0354-446c4b822d1a
• Time: 1/23/2025, 10:06:25 AM
• Cluster URI: https://wabi-south-africa-north-a-primary-redirect.analysis.windows.net/

The Lakehouse creation in the Test workspace starts but fails to complete, leaving a placeholder Lakehouse. This Lakehouse contains critical dependencies like files, folders, and shortcuts required for other artifacts (e.g., Notebooks and Pipelines).

Please give me the related videos and decumentation link to undertand this topic very clearly and also tell me what are the thing that are not supported in the deployment pipeline 

[v-csrikanth · Accepted solution]

Hi @Naman_ 

Currently, Microsoft Fabric deployment pipelines do not fully support copying the contents of Lakehouses, such as tables and files, to the target workspace. While the Lakehouse structure, including schema and metadata, is successfully deployed, the actual data and files are not automatically transferred.

According to the Lakehouse in deployment pipelines documentation, when a deployment occurs for the first time, a new empty Lakehouse with the same name is created in the target workspace.

To populate the Lakehouse with data and files, you can leverage Fabric tools such as data pipelines, notebooks, or manual processes to ensure the target environment contains the necessary content.

Additionally, engineers are working on improve the deployment pipeline experience in Fabric. Below is a new feature on the Roadmap.

Lakehouse Shortcuts metadata on git and deployment pipelines 
https://learn.microsoft.com/en-us/fabric/cicd/best-practices-cicd 

You can also contribute by voting for new features or submitting ideas on Fabric's Idea forum. Here are some related ideas you may find useful:

• Reflect Metadata of Warehouse/Lakehouse in Deployment Pipeline
• Lakehouse views not synced between workspaces via deployment pipelines
• Add Tables or shortcuts to Lakehouse Pipeline deployments in Fabric

If the above information helps you, please give us a Kudos and marked the reply Accept as a Solution.

Thanks,
Cheri Srikanth