topic Run a dataflow gen 2 through api in Dataflow

Run a dataflow gen 2 through api

nokunta — Thu, 01 Aug 2024 21:09:57 GMT

Is it possible to run/refresh a dataflow gen 2 through an api?

I've tried Job Scheduler Api with jobType=dataflowgen2 but with service principal authentication doesn't work.

Is there a workaround?

Re: Run a dataflow gen 2 through api

Anonymous — Fri, 02 Aug 2024 02:51:39 GMT

Currently, there is no suitable api to implement dataflow gen2 refresh. But you can realize data refresh by copy data activity in data pipeline, refer to the following api:

PATCH https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/dataPipelines/{dataPipelineId}

You can refer to below document:

Items - Update Data Pipeline - REST API (DataPipeline) | Microsoft Learn

Best Regards,
Adamk Kong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Re: Run a dataflow gen 2 through api

nokunta — Fri, 02 Aug 2024 10:47:05 GMT

Ok, my dataflow already appends the data to the destination I didn't want to duplicate this data.

Any idea when pipeline type will work with service principal or when we will have an api ro run/schedule dataflows?

Re: Run a dataflow gen 2 through api

frithjof_v — Mon, 05 Aug 2024 22:17:06 GMT

Please vote to highlight the need for service principal authentication in Fabric API:

https://ideas.fabric.microsoft.com/ideas/idea/?ideaid=e5ec086c-f1ce-ee11-92bd-6045bdbce644