Fabric Deployment Pipelines

[hsn367]

I am exploring fabric from data science point of view and was working on deploying sample project to test and prod stages. In my sample solution I have notebooks, data pipeline and lakehouse. When I deployed the solution to test, I deployed these three items. Then when I opened the notebook in test workspace, I see a lakehouse attached to that notebook and I can see all the data in that lakehouse from the side pannel of notebook.

But when I open the lakehouse in a new tab or you can see in explorer, I don't see any data there.

Now in the first image, when from the side pannel I deleted one table in test workspace, the table got deleted from the lakehouse of development workspace as well.
When I deployed the solution to test stage since my lakehouse also got deployed to test stage, I though the lakehouse would have been cloned/copied to tests stage/workspace as well and that both workspaces would have their own copies of the lakehouse. But this does not seem to be the case.

Now here is what the documentation says:

Supported items

When you deploy content from one pipeline stage to another, the copied content can contain the following items:

• Data pipelines
• Dataflows Gen1
• Datamarts
• Lakehouse
• Notebooks
• Paginated reports
• Reports (based on supported semantic models)
• Semantic models (except for Direct Lake semantic models)
• Warehouses

Item properties that are not copied

The following item properties aren't copied during deployment:

• Data - Data isn't copied. Only metadata is copied

Now it says that "Lakehouse" is a supported item but data is not copied. So then what exactly is being deployed to test stage/workspace. Because when I open the test workspace I see all of the contents there e-g:

So in test workspace the same lakehouse appears as we had in the development workspace, so when the documentation says the "data is not copied", so what exactly does this lakehouse represent here?

Just to summarize the question is "When we deploy from one stage to other and we deploy lakehouse as well, what exactly is being deployed, because we can see the same lakehouse in test stage after deployment, so is it just a pointer over the development lakehouse or what?"

[v-cboorla-msft · Accepted solution]

Hi @hsn367 

Thanks for using Microsoft Fabric Community.

Right now, the Lakehouses are created empty. We are working on improving that and getting the definition of the Lakehouse to be exported and deployed. It will come in pieces so it will take time until the full definition will become available.

What we recommend to do with current options are:

• Use data pipelines and notebooks to build the Lakehouse as much as possible. Since they are source controlled and deployed with their definition, you can manage the Lakehouse content and changes by deploying the data pipeline/notebook and running it to apply the changes into the Lakehouse.
• For things that can't be managed through data pipeline/notebook, it will still need to happen manually. It needs to be configured once and then it will not be overridden in later deployments. For example- the shortcut to the bronze LH.

Docs to refer -
Lakehouse deployment pipelines and git integration - Microsoft Fabric | Microsoft Learn
Best practices for lifecycle management in Fabric - Microsoft Fabric | Microsoft Learn

You can refer same thread : Solved: Deployment pipeline and lakehouse content - Microsoft Fabric Community

I hope this information helps. Please do let us know, if you have any further queries.

Thank you.

[v-cboorla-msft]

Hi @hsn367 

Glad that you query got resolved.
Please continue using Fabric Community for further queries.

Thanks.