Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Grow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.

Reply
Braxx99
Frequent Visitor

dataflow incremental refresh slower then full refresh

I have a dataflow with 3 tables. 2 of them have the incremental refresh applied.  

- the dataflow takes about 1:57min to complete with incremental mode

- when incremental refresh is turn off it takes about 48min

That's a problem cos I expected incremental would be faster.

 

Let's take one of these incrementaly refreshed table to examin closely.

- it is configured to refreshed last 45 days, although 48 partitions are created

- the refresh starts at 13:08

- 32 out of 48 partitions are refreshed in parallel starting at 13:08

- rest of te partitions waiting in queue. The last two ones begin refresh at 14:09

 

here is a screenshot from dataflow refresh logs. (not all the columns and partition visible)

Braxx99_0-1707302223340.png

 

questions:

1) why not all of the partitions are refreshed in parallel? 

2) what can I do to be able to refresh all in parallel? I guess this is a problem why the incremental refresh takes so long. The partition which is refreshed at 14:09 need 37min to complete. 

3 REPLIES 3
Braxx99
Frequent Visitor

@v-kongfanf-msft any updates? Thanks in advance

v-kongfanf-msft
Community Support
Community Support

Hi @Braxx99 ,

 

Detailed documentation on incremental refresh and real-time data troubleshooting is officially available. Refer to below document:

Troubleshoot incremental refresh and real-time data in Power BI - Power BI | Microsoft Learn

 

Best Regards,
Adamk Kong

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Thanks for the link.

Unfortunatelly, I could not find answer to my questions there. 

What I am particulary interested in is what are the possible reasons some of the partitions are queued?

 

The source of the table mentioned earier is SQL Server on-premises. It has about 12mln rows and cover daily data from 2019-01-02

Helpful resources

Announcements
RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

MayPowerBICarousel

Power BI Monthly Update - May 2024

Check out the May 2024 Power BI update to learn about new features.

Top Solution Authors