Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
Braxx99
Frequent Visitor

dataflow incremental refresh slower then full refresh

I have a dataflow with 3 tables. 2 of them have the incremental refresh applied.  

- the dataflow takes about 1:57min to complete with incremental mode

- when incremental refresh is turn off it takes about 48min

That's a problem cos I expected incremental would be faster.

 

Let's take one of these incrementaly refreshed table to examin closely.

- it is configured to refreshed last 45 days, although 48 partitions are created

- the refresh starts at 13:08

- 32 out of 48 partitions are refreshed in parallel starting at 13:08

- rest of te partitions waiting in queue. The last two ones begin refresh at 14:09

 

here is a screenshot from dataflow refresh logs. (not all the columns and partition visible)

Braxx99_0-1707302223340.png

 

questions:

1) why not all of the partitions are refreshed in parallel? 

2) what can I do to be able to refresh all in parallel? I guess this is a problem why the incremental refresh takes so long. The partition which is refreshed at 14:09 need 37min to complete. 

3 REPLIES 3
Braxx99
Frequent Visitor

@v-kongfanf-msft any updates? Thanks in advance

v-kongfanf-msft
Community Support
Community Support

Hi @Braxx99 ,

 

Detailed documentation on incremental refresh and real-time data troubleshooting is officially available. Refer to below document:

Troubleshoot incremental refresh and real-time data in Power BI - Power BI | Microsoft Learn

 

Best Regards,
Adamk Kong

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Thanks for the link.

Unfortunatelly, I could not find answer to my questions there. 

What I am particulary interested in is what are the possible reasons some of the partitions are queued?

 

The source of the table mentioned earier is SQL Server on-premises. It has about 12mln rows and cover daily data from 2019-01-02

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors