Memory error during refresh in small dataset for a premium P1 workspace

[DouweAtAlfa]

My dataset refreshes fine when in PowerBI desktop (10min), but after publishing it in a Premium workspace and refreshing (no other refreshes goiing on) it results in the error below:

Fout in de gegevensbron: Resource Governing: This operation was canceled because there wasn’t enough memory to finish running it. Either increase the memory of the Premium capacity where this dataset is hosted or reduce the memory footprint of your dataset by doing things like limiting the amount of imported data. More details: consumed memory MB, memory limit MB, database size before command execution MB. Learn more, see https://go.microsoft.com/fwlink/?linkid=2159753.
URI cluster: WABI-NORTH-EUROPE-K-PRIMARY-redirect.analysis.windows.net
Activiteits-id: befafc18-ef61-4079-a668-ea4c469ad407
Aanvraag-id: 278e0ebe-0564-78c2-debd-cc650d67c76c
Tijd: 2022-09-01 09:52:19Z

Experimenting: I've published the dataset to an workspace PRO (insteadof Premium) and then it refreshes successfully, but taking more then one hour.

Because we do not want to go back to Pro and also do not want to go higher then Capacity P1, can anyone tell how to reduce memory (while is is much smaller then 25Gb)? 

kind regards

Douwe

[DataInsights]

Not sure if this is related, but all of my dataflows sourced from an Azure SQL database started failing as of 2022-09-06 12am PST. Here's the error message from a dataflow (Premium workspace) that normally refreshes in 30 seconds:

"Compute engine failed: Insufficient memory. Please increase your compute engine/workload memory and try again.. Details: A severe error occurred on the current command. The results if any should be discarded."

[RMay]

Yes - we have experienced this too - just started 8 September for us

[Gazmeist]

Us as well - just made this post before seeing yours > https://community.powerbi.com/t5/Issues/Issue-Scheduled-refreshes-failing-due-to-not-enough-memory/i...

[v-mengzhu-msft]

Hi @DouweAtAlfa ,

I investigated your error through this doc:

Troubleshoot XMLA endpoint connectivity in Power BI - Power BI | Microsoft Docs

it says some situation may cause the error as yours, eg:

Through these options maybe can avoid error:

Best regards,

Community Support Team Selina zhu

[DouweAtAlfa]

I've tried to reduce the memory footprint of the dataset by limiting the amount of data loaded with each refresh. The model .pbix was less then 13Mb. Still an error to much memory (25Gb)? On the desktop refresh de proces of msmdsrv.exe never went higher then 2Gb (rule of thumb was 3x 2Gb=6Gb is smaller then 25Gb)?  (No other refreshes runnning at that time.)

Later on I've tried to set the file max parallel processes  to 1 and then it comes through, so I have a workaround now, but is takes 5 times more time from 10min to 50min even with the full dataset?

Did something change in the june release according using memory for premium workspace, is it logging to much, because the PRO workspace could handle the memory ? 

[v-mengzhu-msft]

@DouweAtAlfa ,

The June release does not update the changes to memory usage. Refreshing in the premium workspace can generally be referenced in this document if you encounter limitations, which are not related to the pro workspace.

Power BI Premium Gen2 capacity load, overload, and autoscale - Power BI | Microsoft Learn

You can ensure a smooth refresh by purchasing a larger capacity SKU or using autoscale.

Best regards,

Community Support Team Selina zhu