Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
6 years ago

Dataflows refresh duration very inconsistent and slow

Hey all!   I wonder if anyone else experiences this, whether it is normal. I have a dataset coming in from Google Bigquery, and I run a dataflow to store the data. I've set a refresh schedule, and ...
Dataflow
v-eachen-msft's avatar
v-eachen-msft
Community Support
6 years ago

Hi Anonymous ,

 

You could update your gateway to latest version if your dataflow is running via the on-premise gateway.

 

Anonymous's avatar
Anonymous
Not applicable
6 years ago

Hey Eads,

 

I'm not running a gateway. The data is coming directly from google bigquery over a cloud-to-cloud connection.

 

Jaap

  • v-eachen-msft's avatar
    v-eachen-msft
    Community Support
    6 years ago

    Hi Anonymous ,

     

    If you have a Premium license, you could follow this blog to set dataflow container size:

    https://blog.crossjoin.co.uk/2019/04/21/power-bi-dataflow-container-size/

    If you gave a Pro license, you could try the following workarounds:

    1. You could broke up a big model into entities in 5-6 different dataflows in the same workspace. Then stagger the refresh time.

    2. You could reduce the queries from that biggest table.

     

    • Anonymous's avatar
      Anonymous
      Not applicable
      6 years ago

      Although your suggestions seem sound, the problem is that in powerBI desktop, this data refreshes in 11 SECONDS. So splitting up the dataset doesnt make any sense, it's already really really small, like 1000/2000 rows.

       

      Thanks for thinking along, but I think we're looking in the wrong place. 

       

      Jaap