Inefficient Dataflow default orchestration

[2366]

Hi All,

I have setup a dataflow C which is sourced from A and B where linked entities is turned off by disable load. The transformation steps I have implemented in C takes about 40 minutes to run. I am using power automate to load A and B and trigger C.
however I now wanted to implement a another transformation logic on C by merging another table from D which has now made the data flow run for more than 10 hours.

I turned on linked entities in C and the refresh takes about 30 minutes now.

I understand, powerbi is now using enhanced compute engine, hence the refresh is faster but now the problem I have run into is, C is refreshed 3X times everyday, whenever I refresh A, B and D which is inefficient because my overall runtime is 1.5 hours.
ideally it should be just 40 minutes because A, B, and D load within 8 minutes and C runs for about 30 minutes with enhanced compute engine. 

Shouldn't there be a way, I can configure the dataflow to retain enhanced compute engine but only refresh the final table after all my source dataflows are refreshed?

thanks

[ysusanto]

Hi, I was facing the same issue, what works for me was in the combined table, I unchecked the "Enabled Load" option within the Properties, and schedule refresh time on the Settings. 

[bcdobbs]

Bit of a workaround however...

If you edit the power query in advanced editor for C you will see it references the linked dataflows by GUID (long unique I'd). If you manually edit the code so it navigates by the name of the power query the auto refresh doesn't kick in. Away from a computer at the moment but can supply an example if needed.

[2366]

Thank you @bcdobbs! this is something I have already done by creating parameters for Data flow id which breaks the auto refresh but it just reloads the linked table all over again.

Does anyone ever feel like this is a really inefficient design of dataflows? It makes no sense for the target table to refresh multiple times every time the upstream data source is refreshed unless all the linked sources are refreshed