The ultimate Microsoft Fabric, Power BI, Azure AI, and SQL learning event: Join us in Stockholm, September 24-27, 2024.
Save €200 with code MSCUST on top of early bird pricing!
Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started
I have a dataflow with 3 tables. 2 of them have the incremental refresh applied.
- the dataflow takes about 1:57min to complete with incremental mode
- when incremental refresh is turn off it takes about 48min
That's a problem cos I expected incremental would be faster.
Let's take one of these incrementaly refreshed table to examin closely.
- it is configured to refreshed last 45 days, although 48 partitions are created
- the refresh starts at 13:08
- 32 out of 48 partitions are refreshed in parallel starting at 13:08
- rest of te partitions waiting in queue. The last two ones begin refresh at 14:09
here is a screenshot from dataflow refresh logs. (not all the columns and partition visible)
questions:
1) why not all of the partitions are refreshed in parallel?
2) what can I do to be able to refresh all in parallel? I guess this is a problem why the incremental refresh takes so long. The partition which is refreshed at 14:09 need 37min to complete.
Hi @Braxx99 ,
Detailed documentation on incremental refresh and real-time data troubleshooting is officially available. Refer to below document:
Troubleshoot incremental refresh and real-time data in Power BI - Power BI | Microsoft Learn
Best Regards,
Adamk Kong
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Thanks for the link.
Unfortunatelly, I could not find answer to my questions there.
What I am particulary interested in is what are the possible reasons some of the partitions are queued?
The source of the table mentioned earier is SQL Server on-premises. It has about 12mln rows and cover daily data from 2019-01-02
Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.
Check out the August 2024 Power BI update to learn about new features.
User | Count |
---|---|
53 | |
22 | |
11 | |
10 | |
9 |
User | Count |
---|---|
113 | |
32 | |
30 | |
18 | |
18 |