Incremental refresh is not working during schedule refresh

[borutsustar]

Hi.

I recently implemented incremental refresh on Power BI service. I’ve noticed that incremental refresh doesn’t work with schedule refresh on daily basic, but only with manual refresh (on demand)! This behaviour is very strange. 

I know that every first refresh take longer because it take full refresh of dataset, but each subsequent refresh must be using incremental refresh. 

So, i am wondering why incremental refresh doesn't work during schedule refresh? 

Incremental refresh should be working within Power BI Pro licence. So, i don't see a problem here.

Borut

[v-robertq-msft]

Hi, @borutsustar 

According to incremental refresh, I don’t think it can’t work with scheduled refresh on daily basis. I guess it’s due to the refresh mechanism of the  scheduled refresh:

The target of scheduled refresh is to initiate the refresh within 15 minutes of the scheduled time slot, but a delay of up to one hour can occur if the service can't allocate the required resources sooner.

Therefore, I think you can wait a couple of time (Not over one hour) and go to the “refresh history” to check if the scheduled refresh succeeded:

For further information about data refresh, you can refer the following Microsoft documents that could help you:

https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh

https://docs.microsoft.com/en-us/power-bi/refresh-scheduled-refresh

https://docs.microsoft.com/en-us/power-bi/refresh-data

Best Regards,

Community Support Team _Robert Qin

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[borutsustar]

Thanks for your reply.

I don't have any problem with incremental refresh. It works perfectly fine. I understand the delays during schedule refresh and i have no problem with that. I also understand how incremental refresh works an how it needs to be set up.  But, according to refresh history, incremental refresh does not work during schedule refresh. It works only with manual refresh (refresh on demand). Just to explain why i think so.

Each first refresh of the dataset takes approximately 12-15 minutes. Every time i manual refresh the dataset it takes only 1-2 minutes. During schedule refresh, it takes 12-15 minutes, no matter when it is stared. I've done a lot of tests and i get the same results every time.

I do understand the delays and i also understand that sometimes the refresh of dataset ends sooner, sometimes later, because of shared resources.

But what I don't understand is why manual refresh takes 1-2 minutes, 5 minutes later after schedule refresh starts it takes 12-15 minutes? I find no explanation in this.

There is nothing wrong with incremental refresh. I read a lot of documentations and incremental refresh should be working during schedule refresh, regardless of Power BI Pro or Premium license. Am I wrong?

Is there any other limitation that prevents incremental refresh during schedule (avtomatic) refresh?

[v-robertq-msft · Accepted solution]

Hi, @borutsustar 

According to my research, here’s a detailed explanation about this problem by the technical support of Microsoft, you can take a look:

Based on my research, The reason scheduled refresh takes longer is because PBI smooths scheduled refresh so AS Engine and Gateway is not flooded with request. PBI doesn't do smoothing for on-demand refresh.

Such as if you set scheduled refresh at 9:00 am, it may start at 9:14 am, and end in 9:16 am, so seems it refresh 10 - 15 min, but if execute an on-demand refresh at 9:00 am, it will start right now and may end in only few minutes. This document also mentioned : 

Note also that the configured refresh time might not be the exact time when Power BI starts the next scheduled process. Power BI starts scheduled refreshes on a best effort basis. The target is to initiate the refresh within 15 minutes of the scheduled time slot, but a delay of up to one hour can occur if the service can't allocate the required resources sooner.

What’s more, it’s also a popular issue in the Power BI issue forum:

https://community.powerbi.com/t5/Issues/Why-is-the-On-Demand-Refresh-faster-than-Schedule-Refresh/id...

Therefore, I think it’s not because your incremental refresh not happens, it’s just related to the problem with the refresh mechanism of Power BI. If you need the report to be ready by a certain time, schedule the refresh ahead of time or execute an on-demand refresh.

Best Regards,

Community Support Team _Robert Qin

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.