Forum Discussion
dataflow incremental refresh slower then full refresh
I have a dataflow with 3 tables. 2 of them have the incremental refresh applied.
- the dataflow takes about 1:57min to complete with incremental mode
- when incremental refresh is turn off it takes about 48min
That's a problem cos I expected incremental would be faster.
Let's take one of these incrementaly refreshed table to examin closely.
- it is configured to refreshed last 45 days, although 48 partitions are created
- the refresh starts at 13:08
- 32 out of 48 partitions are refreshed in parallel starting at 13:08
- rest of te partitions waiting in queue. The last two ones begin refresh at 14:09
here is a screenshot from dataflow refresh logs. (not all the columns and partition visible)
questions:
1) why not all of the partitions are refreshed in parallel?
2) what can I do to be able to refresh all in parallel? I guess this is a problem why the incremental refresh takes so long. The partition which is refreshed at 14:09 need 37min to complete.
3 Replies
- AnonymousNot applicable
Hi Braxx99 ,
Detailed documentation on incremental refresh and real-time data troubleshooting is officially available. Refer to below document:
Troubleshoot incremental refresh and real-time data in Power BI - Power BI | Microsoft Learn
Best Regards,
Adamk KongIf this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
- Braxx99Frequent Visitor
Thanks for the link.
Unfortunatelly, I could not find answer to my questions there.
What I am particulary interested in is what are the possible reasons some of the partitions are queued?
The source of the table mentioned earier is SQL Server on-premises. It has about 12mln rows and cover daily data from 2019-01-02
- Braxx99Frequent Visitor
Anonymous any updates? Thanks in advance