Forum Discussion

Filipbukvic96's avatar
Filipbukvic96
Frequent Visitor
1 year ago
Solved

Question about using data flow gen2 (ci/cd) in pipelines as parameters

I want to use dataflow gen2 (ci/cd) as a dynamic value parameter in pipelines but there is no way to forward any kind of unique identifier of a dataflow gen2 (ci/cd). With standard dataflows gen2 I w...
governance
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi Filipbukvic96 ,

    Sorry for the delay in getting back to you. I went through the issue in detail, and the main reason you’re hitting those errors is that Dataflows Gen2 (CI/CD) behave differently compared to standard Gen2 dataflows. When they are Git-linked, the artifact in the workspace is basically read-only, so Fabric APIs can’t refresh them directly that’s why you keep getting InvalidRequest or UnknownException errors.

    Here’s the solution that works for you.

    1.Deploy the CI/CD Dataflow into Live Mode

    Use Deployment Pipelines in Fabric to push the dataflow from the Git branch into the target workspace.

    Once deployed, the dataflow becomes a workspace-level artifact, and refresh works as expected.

    2.Get the GUID dynamically

    Continue using the Fabric REST API:

    GET https://api.fabric.microsoft.com/v1/dataflows

    Filter by name, and grab the id. Trigger the refresh successfully

    Use the refresh endpoint.

    POST https://api.fabric.microsoft.com/v1/dataflows/{dataflowId}/refreshes

    This works for the deployed (live) dataflow, not the Git-linked one.

     

    Why this is the only stable option right now.
    Git-linked dataflows can’t be refreshed via API or pipeline because the workspace version is just a mirror from Git. Deploying it creates a real artifact that the API can work with.

    Regards,
    Akhil.

Anonymous's avatar
Anonymous
Not applicable
1 year ago

Hi Filipbukvic96 ,

Thanks  ibarrau for your inputs. Yes, using the Fabric REST API to list the dataflows and filter by name works perfectly for getting the GUID of a Dataflow Gen2 (CI/CD). I was able to retrieve the ID and pass it as a dynamic parameter in my pipeline. Appreciate the clear pointer to the docs.


Thanks,
Akhil.