Create new connection api for Fabric Data Pipelines

[BramT]

I was already able to create a Web v2 connection with the Fabric api. We use this connection for a monitoring pipeline.

To call the pipeline from another pipeline I need a connection id for Fabric Data Pipelines. The documentation is a bit scarce. I want to use OAuth2 credentials from the same credentials I use with the Fabric API. So far without success. I use the FabricPS-PBIP module from GitHub.

Here is my not working code:

Set-FabricAuthToken -reset
$datapipelineToken = Get-FabricAuthToken

$body = @{
    connectivityType = 'ShareableCloud'
    displayName      = '9A Monitoring'
    connectionDetails = @{
        type           = 'FabricDataPipelines'
        creationMethod = 'FabricDataPipelines.Actions'
        parameters     = @(
            @{
              dataType = 'Text'
                name     = 'path'
                value    = 'FabricDataPipelines'
            }
        )}
    privacyLevel = 'Organizational'
    credentialDetails = @{
        singleSignOnType     = 'None'
        connectionEncryption = 'Encrypted'
        skipTestConnection   = $false
        credentials = @{
            credentialType = 'OAuth2'
            Token = $datapipelineToken
        }
    }
} | ConvertTo-Json -Depth 5
$pipelinesconnectionid = Invoke-FabricAPIRequest -uri "connections" -method Post -body $body
Return $pipelinesconnectionid.id

Can anybody point me in the right direction?

[BramT]

This would solve my issue:
https://community.fabric.microsoft.com/t5/Fabric-Ideas/Managed-Identity-for-Fabric-items/idi-p/47295...

[BramT]

UPDATE: The workspace identity can only be used to authenticate Azure resources, but for another Fabric Data Pipeline in the same workspace the workspace identity cannot be used to authenticate.
@burakkaragoz Can you help me how to authenticate the connection with OAuth2? It can be the same usertoken that is used to connect to the fabric api itself.

$body = @{
    connectivityType  = 'ShareableCloud'
    displayName       = 'Fabric Data Pipelines Connection'
    connectionDetails = @{
        type           = 'FabricDataPipelines'
        creationMethod = 'FabricDataPipelines.Actions'
    }
    privacyLevel      = 'Organizational'
    credentialDetails = @{
        singleSignOnType     = 'None'
        connectionEncryption = 'NotEncrypted'
        skipTestConnection   = $false
        credentials          = @{
            credentialType = 'OAuth2'
            ???? = '????'
            ???? = '????'
        }
    }
} | ConvertTo-Json -Depth 5

[BramT]

Hi @burakkaragoz , 

Thanks for your reply. You pushed me into the right direction. The FabricDataPipelines doesn't really need authentication as it is a pipeline within Fabric calling another pipeline. So to use the workspace authentication in stead was a great idea.

The parameters needed where unclear from the documentation. Finally after simply removing the parameters completely it worked:

$body = @{
    connectivityType  = 'ShareableCloud'
    displayName       = 'Fabric Data Pipelines Connection'
    connectionDetails = @{
        type           = 'FabricDataPipelines'
        creationMethod = 'FabricDataPipelines.Actions'
    }
    privacyLevel      = 'Organizational'
    credentialDetails = @{
        singleSignOnType     = 'None'
        connectionEncryption = 'NotEncrypted'
        skipTestConnection   = $false
        credentials          = @{
            credentialType = 'WorkspaceIdentity'
        }
    }
} | ConvertTo-Json -Depth 5

[v-veshwara-msft]

Hi @BramT ,

Thanks for confirming that it's working now and for sharing the final working code.

Using WorkspaceIdentity is the right approach when triggering one Fabric pipeline from another, since both run within the same context and don't require separate OAuth2 credentials.

Thanks to @burakkaragoz  for pointing out that OAuth2 isn't needed for this type of connection and suggesting workspace authentication instead.

Please reach out for further assistance.

Thank you.

[BramT]

Hello @v-veshwara-msft ,

The final code can create a Fabric Data Pipelines connection with workspace authentication, but unfortunately, the connection cannot be selected in a Fabric pipeline to connect to another Fabric pipeline within the same workspace. The connection needs to be authenticated using OAuth2. I was not able to adjust the code to create the OAuth2 credentials. The issue hasn't been resolved yet and the documentation for the rest api for connections is not clear on how to do this exactly. Can someone from the Microsoft team help me out?

[burakkaragoz]

Hi @BramT ,

Your approach is close, but there are a few issues with the Fabric Data Pipelines connection setup. The main problem is that Fabric Data Pipelines connections don't use OAuth2 tokens the same way external APIs do.

Here's what needs to change:

For Fabric Data Pipelines, use service principal authentication instead:

$body = @{
    connectivityType = 'ShareableCloud'
    displayName      = '9A Monitoring'
    connectionDetails = @{
        type           = 'FabricDataPipelines'
        creationMethod = 'Manual'
        parameters     = @(
            @{
                dataType = 'Text'
                name     = 'workspaceId'
                value    = 'your-workspace-id'
            }
        )
    }
    privacyLevel = 'Organizational'
    credentialDetails = @{
        singleSignOnType     = 'None'
        connectionEncryption = 'Encrypted'
        skipTestConnection   = $false
        credentials = @{
            credentialType = 'ServicePrincipal'
            servicePrincipalId = 'your-service-principal-id'
            servicePrincipalKey = 'your-service-principal-secret'
        }
    }
} | ConvertTo-Json -Depth 5

Alternative - use the simpler approach:

Since you're calling one Fabric pipeline from another, you might not need a separate connection at all. Try using the Execute Pipeline activity directly with workspace authentication.

Debugging tips:

• Check what error you're getting from the API call
• Try skipTestConnection = $true to bypass connection validation
• Make sure your service principal has proper permissions on both workspaces

What specific error are you seeing when you run your current code? That'll help narrow down the exact issue.

---

If my response resolved your query, kindly mark it as the Accepted Solution to assist others. Additionally, I would be grateful for a 'Kudos' if you found my response helpful.
This response was assisted by AI for translation and formatting purposes.