Fabric Data Engineering and Deployment Pipelines

[mark_carlisle]

Been exploring Fabric, specifically the Lakehouse, Warehouse, Pipelines, and Dataflows (Gen 2). I've built a Pipeline that

• Gets JSON data from SFTP and lands that in a Lakehouse
• Triggers a Dataflow to flatten that data and lands the output into a Warehouse

I done all this in a Dev workspace and made the assumption that I would be able to use deployment pipelines to move these artefacts to the Test workspace and then on to Prod in exactly the same way I would do with a Power BI dataset or dataflow and found that this is not possible.

I'm not able to find any release plan info on Fabric so my questions for this forum are:

• Are there any release plan info?
• Are there any plans to enable deloyment pipelines for these Fabric artefacts?
• If there are no plans is this because Lakehouse, Warehouse, Pipelines, and Dataflows (Gen 2) should not be developed in this way and if thats the case then is there any documentation or advice that anyone can offer on how development of these artefacts should take place without impacting production environments.

[v-nikhilan-msft]

Hi ,
Thanks for using Microsoft Fabric Community.As I understand you are trying to work with deployments, currently deployments in Fabric are having few limitations.

As per the official document there are some items which are supported and some items which are unsupported. I have provided the link for your reference.

https://learn.microsoft.com/en-us/fabric/cicd/deployment-pipelines/understand-the-deployment-process...

There is no current release plan information for deployment pipelines for Fabric artifacts. However, the Microsoft Power BI team is aware of the demand for this feature and is considering it for future development.

In the meantime, there are a few workarounds that you can use to deploy Fabric artifacts to Test and Prod environments:

1) Use a manual deployment process: This can be done by manually copying the Fabric artifacts from the Dev workspace to the Test and Prod workspaces.
2) Use a third-party CI/CD tool: There are a number of third-party CI/CD tools that support the deployment of Fabric artifacts. For example, you could use Azure DevOps to create a pipeline that deploys your Fabric artifacts to Test and Prod environments.
3) Use a custom solution: You could develop your own custom solution to deploy Fabric artifacts to Test and Prod environments. This could involve using the Power BI REST API or the Service Fabric SDK.

Appreciate if you could share the feedback on our feedback channel, which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

Feedback Link:  https://ideas.fabric.microsoft.com/

I hope this information is helpful. Please let me know if you have any other questions.

[v-nikhilan-msft]

Hi @mark_carlisle ,
Following up to see if the given suggestion was helpful. And, if you have any further query do let us know.

[mark_carlisle]

Can you point me in the direction of some documentation on point 2 - Use a third-party CI/CD tool.

[v-nikhilan-msft · Accepted solution]

Hi @mark_carlisle ,
I have reached the internal team regarding the unsupported items in the deployment and they have updated me that there is no alternative approach to deploy unsupported items in Deployment pipelines. However, the team is working on adding their artifacts to Deployment pipelines these days and some are planned to be supported by GA. Hence I request you to wait for the new version gets deployed.
Thanks for understanding.

[v-nikhilan-msft]

Hi @mark_carlisle ,
Following up to check whether your query got resolved. If you have any further queries please do let us know.

[mark_carlisle]

I've raised an idea as per your suggestion - Microsoft Idea.

Thanks

[v-nikhilan-msft]

Hi @mark_carlisle ,

Thanks for raising your concern and I am glad your query got resolved. Please continue using Fabric Community for your queries.