Forum Discussion
Need help: Dynamically updating parameter values during Fabric CI/CD deployment
You need to update parameter values when deploying Data Pipelines and Notebooks across Fabric environments using Azure DevOps or REST APIs. This is a common CI/CD challenge.
The Fabric REST API doesn't natively support parameter overrides during deployment. You need a post-deployment step to update parameters.
Here's your approach:
Option 1: Use Fabric REST API for Parameter Updates
After deploying your pipeline, call the Update Item Definition API to modify parameters:
$workspaceId = "your-workspace-id"
$pipelineId = "your-pipeline-id"
$bearerToken = "your-token"
$body = @{
definition = @{
parts = @(
@{
path = "pipeline-content.json"
payload = @{
activities = @(...)
parameters = @{
connectionId = @{
type = "String"
defaultValue = "prod-connection-id"
}
databaseName = @{
type = "String"
defaultValue = "ProductionDB"
}
}
}
}
)
}
} | ConvertTo-Json -Depth 10
Invoke-RestMethod -Uri "https://api.fabric.microsoft.com/v1/workspaces/$workspaceId/items/$pipelineId/updateDefinition" -Method Post -Headers @{Authorization = "Bearer $bearerToken"} -Body $body -ContentType "application/json"
```
**Option 2: Environment-Specific Definition Files**
Store separate pipeline definition JSON files for each environment in your repo:
```
/pipelines
/dev
pipeline-config.json
/test
pipeline-config.json
/prod
pipeline-config.jsonDuring deployment, reference the correct file based on your target environment variable.
Option 3: Notebook Parameters Workaround
For Notebooks specifically, use Azure DevOps variable groups and pass parameters at runtime:
# In your notebook
dbutils.widgets.text("environment", "dev")
dbutils.widgets.text("connectionString", "default")
env = dbutils.widgets.get("environment")
connString = dbutils.widgets.get("connectionString")Then schedule or trigger the notebook with environment-specific parameter values through the Fabric API.
Option 4: Post-Deployment Script
Create a PowerShell script in your pipeline that reads environment variables and updates Fabric items:
- task: PowerShell@2 inputs: targetType: 'inline' script: | $env = "$(Environment)" $configFile = Get-Content "./config/$env.json" | ConvertFrom-Json # Update pipeline parameters via REST API # Call Update Definition API with environment-specific values
The GitHub sample you referenced focuses on deployment but doesn't handle parameter substitution. You need to add a custom step after the deployment task.
- CloudVasu10 months ago
Helper I
Thanks a lot for the detailed explanation, this is really helpful.
That makes sense now. The Fabric REST API doesn’t currently support direct parameter overrides during deployment, so adding a post-deployment step to update pipeline definitions using the updateDefinition API looks like the right approach.
I’ll try implementing Option 1 (calling the Update Definition API after deployment) in my Azure DevOps YAML pipeline to dynamically replace connection IDs and other environment-specific parameters.
Also, good point about maintaining separate config JSON files or using environment variables for notebooks. That gives us flexibility for future scaling.
I really appreciate the clarification, this clears up a blocker we were facing. I’ll implement this approach and share my updates here once tested