Best Way to do CICD for data Science workloads

[Srisakthi]

Hello Everyone,

As Microsoft Fabric Deployment pipleine doesnt support CICD for data science workloads. Can someone guide me the best way to do CICD for data science workloads using Azure devops pipeline or any other tool?

also is it good to have separate repository for each data science workspace ?

Regards,

Sri

[v-lgarikapat]

Hi  @Srisakthi ,

Thanks for reaching out to the Microsoft fabric community forum

@SamsonTruong 

Thanks for your prompt response

@Srisakthi 

I wanted to follow up and confirm whether you’ve had the opportunity to review the information  provided by  @SamsonTruong  . If you have any questions or need further clarification, please don’t hesitate to reach out.

We appreciate your collaboration and support!

Best regards,
Lakshmi.

[v-lgarikapat]

Hi @Srisakthi ,

We’d like to confirm whether your issue has been successfully resolved. If you still have any questions or need further assistance, please don’t hesitate to reach out. We’re more than happy to continue supporting you.

We appreciate your engagement and thank you for being an active part of the community.

Best Regards,
Lakshmi.

[SamsonTruong]

Hi @Srisakthi ,

Which data science workloads are you leveraging? For CI/CD in Fabric, there are a few methodologies you can use. The one I have used is connecting Azure DevOps to a development workspace and leveraging the built-in Fabric Deployment Pipelines as a release pipeline to subsequent environments (test/prod). Here are some additional documentation on methodologies here: https://learn.microsoft.com/en-us/fabric/cicd/manage-deployment

If this helped, please mark it as the solution so others can benefit too. And if you found it useful, kudos are always appreciated.

Thanks,

Samson

Connect with me on LinkedIn

Check out my Blog

Going to the European Microsoft Fabric Community Conference? Check out my Session

[Srisakthi]

Hi @SamsonTruong ,

The link which you have shared is a generic Fabric CICD. I'm looking for is Data Science Workloads. 

To give you high level information of data science workload is all about creating deploying ML models, data agents etc.

Regards,

Srisakthi

[SamsonTruong]

Hi @Srisakthi ,

depending on which data science workloads you are using, the Fabric CICD is still a viable option. Using the built in Fabric deployment pipelines, you have support for notebooks, ML experiments, and ML models. Can you please provide additional information into which specific data science workloads you are working with, as the recommended CICD infrastructure may change based on your use case.

If this helped, please mark it as the solution so others can benefit too. And if you found it useful, kudos are always appreciated.

Thanks,

Samson

Connect with me on LinkedIn

Check out my Blog

Going to the European Microsoft Fabric Community Conference? Check out my Session

[Srisakthi]

Hi @SamsonTruong ,

Thanks for your response. It helps me. My scenario is i have multiple datascience workpsaces and each has ML models. i need to promote all these ML models from different workspace to one workspace in my test environment. What is the best approach i can chose?

Regards,

Sri

[SamsonTruong · Accepted solution]

Hi @Srisakthi , for ML Models, you have 2 options. You can use the built-in Fabric deployment pipelines or you can leverage a Git integration to use a service such as Azure DevOps. Both are fully supported. Here is some documentation below:

• https://learn.microsoft.com/en-us/fabric/data-science/machine-learning-artifacts-git-deployment-pipe...
• https://learn.microsoft.com/en-us/fabric/cicd/git-integration/intro-to-git-integration?tabs=azure-de...