Forum Discussion
Refresh Multiple Dataflows and One Dataset Sequentially
Doesn't have to be repetitive. Each dataflow can be updated as needed. Technically you can set up the action only for dataflow 3 -> dataset 1 and dataset 4 -> dataset 2, and then a separate flow that refreshes the dataflows in sequence.
This is all "event driven" (*) so no repetition.
*) Microsoft marketing speak for incessant polling.
lbendlin If I understand correctly, this approach runs the refreshes of the dataflows in sequence but it seems that the more ideal approach is to run all of the data flows in parallel - this is the value of cloud architecture - and then refresh the dataset only after all of the dataflows have refreshed successfully. Running them in sequence will cause delays in processing the dataset and then in turn delay presenting data to end users. Is there an approach in dataflows or Power Automate to run them in parallel?
It depends. Most of all it depends on the performance of your data source. If you bombard a single processor SQL server with more than a handful of concurrent queries it will fall over, for example.
In Power Automate you can have parallel branches of flows, but you would still need to capture the result of the dataflow refresh somehow - and decide what to do in case of a refresh failure, for example.
I would recommend you write down the process logic you want to implement, have it reviewed/critiqued by a colleague or two (including all crazy scenarios you can think of) , and then implement the flow based on that logic.
Thanks lbendlin - yeah, the data source is not compute resource constrained and is scalable for parallel processing. I will look at the parallel Power Automate flow branching. Ideally, there will be a Power Automate component that triggers the refresh of the data flow but also waits until it receives a refresh complete response from the data flow (subject to a timeout limit, which should then cause an error in the Power Automate flow and stop processing).
Ideally, there will be a Power Automate component that triggers the refresh of the data flow but also waits until it receives a refresh complete response from the data flowThere is not. You need to write that part yourself.