Forum Discussion
Parameterized connections in Data pipelines
- 1 year ago
Hello v-saisrao-msft ,
sorry, but your answer doesn't help me.
I don't know if it is working or not, and it is no documentation how it works...
For all others who want to use the feature: For my case i works, when i use the connection id as parameter in "connection". To get the connectionid, I stored the fixed connection and found the id in the json.
we also on the way to leverage that new feature...following findings (without documentation):
Creation/Development Phase
1. Yes, you need to specify the ID, you can get the ID from GatewayMgt>Connections, of the connection as dynamic content.
2. When you create a dynamic connection, dedendant on "connection type", additonal fields need to be provided:
2.1 Type "Fabric SQL Database" = WorkspaceID and SQLDatabaseID needed
2.2 Type "SQL Server" = DatabaseName?
Deployment Phase
Of course making connections dynamic is only one half of the story, the other half is to deploy this pipelines to other workspaces and run them with other parameter values...
Findings:
1. So if you have connections which need "WorkspaceID" you can leverage the system.workspaceID variable to make the parameter value dynamic across workspaces
2. The ConnectionID seems, at least for Fabric-conncetion-types like WH,LH,Database, the same accross workspaces.
3. For other connection-types you need a way to centrally manage different ConnectionIDs per Workspace
ISSUE:
The "THREAT AS NULL" settings are not deployed with Fabric Deployment Pipelines for our scenario 😞
Repro Steps:
1. Create a DEV and PROD Workspace
2. in the DEV workspace create a pipeline1 with a "Lookup" Activity to call a SQL Server Stored Procedure. Make the Connection for the DB with the proc dynamic.
- The SQL Proc needs to have multiple input parameters which are optional, so they need to be flagged in pipeline.LookupActivity.settings.parameter as "TREAT AS NULL"
- If you now deploy pipeline1 from DEV to PROD Workspace with Fabric Pipeline the "THREAT AS NULL" info get lost. If you open pipeline Editor>View>EDIT JSON Code we can see the difference causing the issue:
DEV Pipeline:
PROD Pipeline:
The current workaround is to update the pipelines in PROD after deployment manually , but this is not how it shoud be.... can someone from microsoft qualify that as an bug?