Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Execute Dataflow Tables/Queries in Parallel

The dataflow refresh history allows you to see the execution times by table including start and end time. You can clearly see that tables are executed sequentially. I have not found any option which allows you to execute in parallel as you can in a dataset. Hopefully, this is something the Microsoft Power BI team will address.

 

Requested onDataflow nameDataflow refresh statusTable nameRefresh statusStart timeEnd timeDuration
10/13/2023 3:42Common DataflowCompletedCalendarCompleted10/13/2023 3:42:0210/13/2023 3:42:3300:31.1
10/13/2023 3:42Common DataflowCompletedCustomerCompleted10/13/2023 3:42:3310/13/2023 3:43:0400:30.5
10/13/2023 3:43Common DataflowCompletedItemCompleted10/13/2023 3:43:0410/13/2023 3:43:3500:30.6
Dataflow

4 Replies

  • edhans's avatar
    edhans
    Community Champion
    2 years ago

    Those are really small. I've got some dataflows with long running times and there is some concurrency.

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      I gave those as an example. The entire dataflow takes 45 minutes with the longest table taking 11 minutes.  When you say you have some concurrency, what do you mean and was that something you controlled?

      • edhans's avatar
        edhans
        Community Champion
        2 years ago

        Meaning I've seen times during the refresh where multiple tables/entities are refreshed in parallel.

         

        But you have no control. It does what it does, and concurrency might be a factor of the license/capacity, dependencies, etc.

  • asocorrocognex's avatar
    asocorrocognex
    Helper II
    1 year ago

    Do all the tables depend on the same query?  For example, do you have a query Q that is the starting point for all the others?  That would cause sequential executions.