Deployment of notebooks and data agent via Fabric pipelines

[sahinad]

Hi,

We are trying to deploy our notebooks and data agents via Fabric deployment pipelines. We have two workspaces: Core [Test] and Core.

We couldn't find a way to configure the data source in the notebooks and data agents during the deployment. They still point to the lakehouse in the Core [Test] workspace after the deployment.

Is there a solution for this?

Thanks.

[v-aatheeque]

Hi @sahinad 

Have you had a chance to look through the responses shared earlier? If anything is still unclear, we’ll be happy to provide additional support.

[sahinad]

I've found the way to configure it for notebooks but can't see any option for data agents.

It was also interesting to figure out that you have to be the owner of the items that you want to deploy.

[v-aatheeque · Accepted solution]

Hi ,
Deployment rules are currently supported only for specific item types (e.g., notebooks), and the fabric data agents is in preview state.(https://learn.microsoft.com/en-us/fabric/data-science/how-to-create-data-agent)

You must be the Item Owner to deploy through Fabric deployment pipelines. Workspace Member or Contributor access alone isn’t sufficient.

[v-aatheeque]

Hi @sahinad 

We wanted to follow up to check if you’ve had an opportunity to review the previous responses. If you require further assistance, please don’t hesitate to let us know.

[Hyeen24]

You can configure it in the deployment pipeline itself where you specify the source and destinations data source for each item.

However I find it cumbersome (especially when you have many items to configure), so I do prefer to use a notebook to run and retrieve the environment variables itself such as lakehouse path and other variables

[Mauro89]

Hi @sahinad,

I agree with @tayloramy regarding Data Agent. But for notebooks there are deployment rules available to switch between your data sources. Also the way to use the Fabric CLI commands is possible. 
Here the links to the docs:

• fabric-cicd documentation

• Parameterization - fabric-cicd - for example of parameterization, and Examples - fabric-cicd for example file.

• https://learn.microsoft.com/en-us/fabric/data-engineering/notebook-source-control-deployment

Best regards!

PS: if you find this post helpful, consider leaving kudos and mark it as solution. 

[ChielFaber]

Haven't done this myself but check out the answer to this post:

https://community.fabric.microsoft.com/t5/Fabric-platform/Changing-the-path-in-my-notebook-in-a-depl...

[sahinad]

My problem is different. They struggled with switching credentials from DEV to PROD. In my case, I cannot point the notebooks to the lakehouse on the prod workspace. They still point to the test workspace.

[tayloramy]

Hi @sahinad, 

Deployment pipelines don't support rules for these items, so I recommend trying to use variable libraries in the workspaces and define the proper connections in the variables, then reference those varaiables in your notebooks and data agents. 

 Get Started with Variable Libraries - Microsoft Fabric | Microsoft Learn

Fabric Application Lifecycle Management Variable Library - Microsoft Fabric | Microsoft Learn

If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.