Forum Discussion
Is dataflowgen2 better in performance and refresh than dataflow gen 1?
Hi pallavi_r ,
Thank you andrewsommer for the helpful response!1.Yes, switching to Dataflow Gen 2 may reduce refresh times compared to Gen 1, depending on data size, network, and source performance. To get the best results, ensure query folding is maintained and avoid heavy transformations in the query steps.
2.Moving to Premium Capacity adds longer refresh windows and provides more stable performance. The main benefit is avoiding refresh failures and handling larger datasets more reliably.
If you cannot break the data further, also consider Incremental Refresh in Gen 2 if your SharePoint data has a timestamp. This might reduce ongoing refresh times by 50–80%.
Please refer the attached documents for more understanding:
https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-incremental-refresh
https://learn.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-understand-optimize-refresh
https://learn.microsoft.com/en-us/fabric/data-factory/dataflows-gen2-overview
I hope this helps.If so,give us kudos and consoder accepting it as solution.
Regards,
Pallavi G.
Yes, you should see performance improvement after switching to a Dataflow Gen 1; the trade off is they consume more CUs. A dataflow Gen2 will most likely reduce refresh times.
Yes, moving to a premium capacity will also improve your refresh times and improve stability. The max refresh time for shared (pro) capacity is 2 hours; however it is 5 hours with a premium (P/F SKU).
If your SharePoint source includes a timestamp you could investigate incremental refresh to reduce refresh time.
Please mark this post as a solution if it helps you. Appreciate Kudos.
Hi andrewsommer ,
Thanks for reply. Did you mean moving to dataflow gen 2 will reduce the refresh time to some extent? For example if in dataflow gen 1, it takes 1 hour, then gen 2 refresh time will be less than 1 hour. This is something I want to know. And the same question is for moving to premium capacity. Will it significantly reduce the refresh time?
Thanks,
Pallavi
Hi pallavi_r ,
Thank you andrewsommer for the helpful response!1.Yes, switching to Dataflow Gen 2 may reduce refresh times compared to Gen 1, depending on data size, network, and source performance. To get the best results, ensure query folding is maintained and avoid heavy transformations in the query steps.
2.Moving to Premium Capacity adds longer refresh windows and provides more stable performance. The main benefit is avoiding refresh failures and handling larger datasets more reliably.
If you cannot break the data further, also consider Incremental Refresh in Gen 2 if your SharePoint data has a timestamp. This might reduce ongoing refresh times by 50–80%.
Please refer the attached documents for more understanding:
https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-incremental-refresh
https://learn.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-understand-optimize-refresh
https://learn.microsoft.com/en-us/fabric/data-factory/dataflows-gen2-overview
I hope this helps.If so,give us kudos and consoder accepting it as solution.
Regards,
Pallavi G.
