Forum Discussion
Fabric Warehouse component Deployment using Azure Repo(DevOps)
- 1 year ago
Thanks for sharing your YAML and more details! You're on the right track, and it's great to see your pipeline taking shape. Let me help clarify a few points and share some practical steps and sample YAML to get you going, especially around subscription handling and modular deployments.
1. Azure Subscription in YAML:
To use Azure CLI tasks in your pipeline, you need to provide a service connection that has access to the right Azure subscription. In Azure DevOps, this is handled via a “service connection” (not just the subscription ID).- In your YAML, instead of manually specifying the subscription ID as a variable, use the azureSubscription input, which references a service connection you create in Azure DevOps (Project Settings > Service connections).
- Example:YAML
- task: AzureCLI@2 inputs: azureSubscription: 'Your-Service-Connection-Name' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | # Your scripts here
- The service connection handles authentication and securely stores credentials.
2. Modular/Single & Multi-Select Deployments:
You can use pipeline variables or parameters to control which tables or components deploy, making your pipeline flexible for single or multi-table deployment.- Define a parameter at the top of your YAML:YAML
parameters: - name: tablesToDeploy type: object default: ['Table1', 'Table2']
- Reference this parameter in your deployment scripts to loop through and deploy only the selected tables.
3. Sample Step for Validating Deployment:
Here's how you might add a validation step for your Fabric SQL endpoint: yaml - task: AzureCLI@2 inputs: azureSubscription: 'Your-Service-Connection-Name' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | az login --service-principal -u $(clientId) -p $(clientSecret) --tenant $(tenantId) sqlcmd -S $(sqlEndpoint) -d $(databaseName) -U $(sqlUser) -P $(sqlPass) -Q "SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLES"- Make sure your pipeline variables (like clientId, clientSecret, etc.) are set using Azure DevOps secrets for security.
4. Step-by-Step:
- Set up an Azure Service Connection in Azure DevOps for your subscription.
- Reference this connection in your YAML (azureSubscription: ...).
- Use parameters/variables to control which components get deployed.
- Add validation steps as needed (as above).
- For multi-table or environment-based deployment, loop through parameterized table names or environment settings.
If you want, I can share a more complete YAML template that includes parameterized deployment and best practices for secrets/connection handling. Just let me know your exact scenario (e.g., number of environments, single vs. multiple tables, etc.), and I can tailor it for you.
Summary:
- Use Azure DevOps “service connections” for subscription/auth.
- Parameterize your pipeline for flexible deployment.
- Use inline scripts for validation and deployment logic.
- Secure your secrets with pipeline variables.
Let me know if you’d like a full example YAML or need help with a specific step! You’re making great progress—just a couple of tweaks and you’ll have a robust deployment pipeline.
Hi burakkaragoz , Thank you very much for the response. I have build the pipeline but not able to add azure subscription to it and not sure from where I should take it. I am struggling go ahead of this as of now. Below is the YAML that I have created(Not sure if its correct). It will be greatful if you can share step by step process and YAML examples for single and multiselect deployment. I have looked for similar content on the Web but nothing much available. Below is the YAML;
- burakkaragoz1 year agoSuper User
Thanks for sharing your YAML and more details! You're on the right track, and it's great to see your pipeline taking shape. Let me help clarify a few points and share some practical steps and sample YAML to get you going, especially around subscription handling and modular deployments.
1. Azure Subscription in YAML:
To use Azure CLI tasks in your pipeline, you need to provide a service connection that has access to the right Azure subscription. In Azure DevOps, this is handled via a “service connection” (not just the subscription ID).- In your YAML, instead of manually specifying the subscription ID as a variable, use the azureSubscription input, which references a service connection you create in Azure DevOps (Project Settings > Service connections).
- Example:YAML
- task: AzureCLI@2 inputs: azureSubscription: 'Your-Service-Connection-Name' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | # Your scripts here
- The service connection handles authentication and securely stores credentials.
2. Modular/Single & Multi-Select Deployments:
You can use pipeline variables or parameters to control which tables or components deploy, making your pipeline flexible for single or multi-table deployment.- Define a parameter at the top of your YAML:YAML
parameters: - name: tablesToDeploy type: object default: ['Table1', 'Table2']
- Reference this parameter in your deployment scripts to loop through and deploy only the selected tables.
3. Sample Step for Validating Deployment:
Here's how you might add a validation step for your Fabric SQL endpoint: yaml - task: AzureCLI@2 inputs: azureSubscription: 'Your-Service-Connection-Name' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | az login --service-principal -u $(clientId) -p $(clientSecret) --tenant $(tenantId) sqlcmd -S $(sqlEndpoint) -d $(databaseName) -U $(sqlUser) -P $(sqlPass) -Q "SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLES"- Make sure your pipeline variables (like clientId, clientSecret, etc.) are set using Azure DevOps secrets for security.
4. Step-by-Step:
- Set up an Azure Service Connection in Azure DevOps for your subscription.
- Reference this connection in your YAML (azureSubscription: ...).
- Use parameters/variables to control which components get deployed.
- Add validation steps as needed (as above).
- For multi-table or environment-based deployment, loop through parameterized table names or environment settings.
If you want, I can share a more complete YAML template that includes parameterized deployment and best practices for secrets/connection handling. Just let me know your exact scenario (e.g., number of environments, single vs. multiple tables, etc.), and I can tailor it for you.
Summary:
- Use Azure DevOps “service connections” for subscription/auth.
- Parameterize your pipeline for flexible deployment.
- Use inline scripts for validation and deployment logic.
- Secure your secrets with pipeline variables.
Let me know if you’d like a full example YAML or need help with a specific step! You’re making great progress—just a couple of tweaks and you’ll have a robust deployment pipeline.