Forum Discussion

kadirkkkk1's avatar
kadirkkkk1
Frequent Visitor
11 months ago
Solved

Dataflow Gen1 data refresh duration is allways more than 30 seconds

In dataflow gen1 when i look at refresh history i see each of my table takes at least 30 seconds of update time. It doesn't matter how light is the query is, it always takes tthat much time. If i hav...
Need Help
  • AntoineW's avatar
    AntoineW
    11 months ago

    I understand, but rest assured, your queries are fine it’s just Gen1’s design.

     

    It’s not necessarily a best practice to create many small Dataflows Gen1 if each contains only one or two very light tables, because each one will incur that same startup overhead.

     

    Instead, you’ll get better performance by grouping related tables into a single dataflow so they refresh together within the same container session.

     

     

    Hope it helps you ! 
    Best regards,
    Antoine
AntoineW's avatar
AntoineW
Icon for Super User rankSuper User
11 months ago

Hello kadirkkkk1,

 

A bit weird but it may have an explication, the refresh duration in Dataflow Gen1 includes not just the query execution time, but also service overhead from the Power BI dataflow engine.

 

What you can do : 

 

  • If you’re migrating or optimizing, consider Dataflow Gen2 in Fabric, which uses a newer backend with parallel refresh, lower overhead, and shorter startup times. 

  • Microsoft is investing a lot in this new tool, you can easily migrate and optimize with Dataflow Gen2.

 

Source : 

 

- https://learn.microsoft.com/fr-fr/fabric/data-factory/dataflows-gen2-overview

- https://learn.microsoft.com/fr-fr/fabric/data-factory/move-dataflow-gen1-to-dataflow-gen2

 

Hope it can help you ! 

Best regards,

Antoine