Forum Discussion
How to execute multiple dataflows(Dataflow Gen2 CI/CD Enabled) dynamically in Fabric?
Hello rohit_motwani ,
Yes, you can trigger dataflows with the REST API without any problems.
https://learn.microsoft.com/en-us/rest/api/power-bi/dataflows/refresh-dataflowHowever, I don't think it's the Data Pipeline Orchestration that consumes a lot (it does nothing except trigger dataflows), but data flows are not the fastest elements.
Transferring data directly via a Data Pipeline (copy data), a Python script or other method will be much more efficient.
However, it depends on the context. Dataflows are very practical, but when there is a lot of data, they consume a lot of resources.
Feel free to give me a kudo and accept my answer as the solution if it suits you.
Have a nice day,
VivienHi rohit_motwani,
In my environment, I just add the dataflow to the pipeline and run everything through the pipeline API. This gives me a more stable system that requires less changes as there is only the one API I am interacting with, instead of also interacting with the notebook and dataflow APIs respectively.
That being said, there is an API available for dataflows: Dataflows - Refresh Dataflow - REST API (Power BI Power BI REST APIs) | Microsoft Learn
You should be able to implement it much the same way as the pipeline API.
If you found this useful, consider giving Kudos. If I solved your problem, mark this post as the solution.
Hi tayloramy ,
Thanks a lot for sharing this! Yes, the code you provided works well for executing a pipeline from a notebook. My query was more around whether a similar approach is possible for executing Dataflows using Python. Could you confirm if Dataflow execution is supported via the API, or if it’s currently limited to Pipelines?
Hi rohit_motwani,
In my environment, I just add the dataflow to the pipeline and run everything through the pipeline API. This gives me a more stable system that requires less changes as there is only the one API I am interacting with, instead of also interacting with the notebook and dataflow APIs respectively.
That being said, there is an API available for dataflows: Dataflows - Refresh Dataflow - REST API (Power BI Power BI REST APIs) | Microsoft Learn
You should be able to implement it much the same way as the pipeline API.
If you found this useful, consider giving Kudos. If I solved your problem, mark this post as the solution.