Dataset refresh on Power BI Service loop 4x starting March 2024

[nandic]

Hi,

Starting on March 2024, i saw strange behaviour on Power BI dataset refresh on Power BI Service.
Even when data source is missing needed information (example, column or table have been deleted/renamed), it is trying for 4 times to refresh data. 

On other datasets we see pattern, it is trying to refresheh for 5 minutes, returns timeout error and tries again for additional 3 times.

We haven't changed datasets (data or structure), but we keep having these issues.
After contacting MS support they said that nothing has changed in dataset refresh logic.
Has anyone else had this issue?

Best regards,
Nemanja Andic

[nandic]

First of all, @lbendlin congrats! I have been active on this forum a few years ago and i remember your solutions. 
And you are still active, answering questions 🙂

We have created a ticket a month ago, but they provide no answers, only what is in the documentation.
But our questions would be solved by looking at logs, not documentation.
However, to get such support, i suppose we need to pay premium support which would cost more than year of premium capacity 🙂

[nandic]

@lbendlin thank you for prompt response.
Yes, i found documentation that we now have more detailed refresh log, but it has weird impact on our performance.
For big datasets (a few gigabytes), if there is an error, it is trying 4 times to refresh it which leads to hours of trying.
Main issue is that starting with March 2024, we get frequently timeout issue, but it is random time. Sometimes timeout after 2 minutes, sometimes after 10 minutes, sometimes after 15 minutes.. so it seems that timeout limit is not constant value?
For 2 years dataset refresh took 1.5 hours. And in last month it took from 50 minutes to 5 hours. We lost control on refreshes.

[lbendlin · Accepted solution]

To make matters worse, in the Fabric Capacity costing these will all be included as consuming CUs. I would agree that this is an undesirable behavior especially for larger dataflows that have the potential to run for many hours. We just implemented an internal process to kneecap dataflows at 5 hrs because we were bitten by that issue one too many times.

Since you have a Pro license you can open a Pro ticket at https://admin.powerplatform.microsoft.com/newsupportticket/powerbi

[lbendlin]

There was indeed a change in the retry behaviour a while back (including more detailed logging).  But in your case the error was not recoverable so all retries were exhausted.