Unable to refresh dataset due to "not enough memory to complete this operation".

[kr77025]

Unable to refresh a dataset from the Dataset page on the PBI service. The PBIX file is around 540Mb on our laptop. The workspace has 9Gb+ of available memory. But starting last night we are unable to refresh the dataset with message below:

Processing error.There's not enough memory to complete this operation. Please try again later when there may be more memory available. Memory error: Memory Allocation failure : The paging file is too small for this operation to complete. . Try simplifying or reducing the number of queries.

Cluster URIWABI-US-NORTH-CENTRAL-redirect.analysis.windows.net

Activity IDb5808310-5d56-4a86-9101-4d2f673c93bd

Request ID1a75fbe5-1eff-3de9-e1de-eb1decad3a06

Time2019-04-23 16:21:51Z

Single datasource is from an Azure SQL DW. Biggest table has 70M rows in it. File size of PBIX is 540Mb (containing all data). Vertipaq analyzer report shows it takes up 1.3 Gb of memory. 

Just not able to refresh the dataset on the service.

[GilbertQ]

Hi there

There is a data size limit as far as I am aware when moving data into the Power BI Service which is 1GB, this is before the compression takes place.

Could you possibly see if the source has gotten more rows?

[kr77025]

Actually, i started getting the error now on PBI Desktop as well. After reading in the data, I see (from task manager) that the memory shoots up to 12 Gb and then goes back down to the typical 1.5 Gb. 

I'm able to process 20M rows but not 30M rows, even though i see the memory spike occur.

I was able to reduce the total number of records from 70M (which btw processed fine a couple of days ago) down to 40M. But now we can't refresh this in either the service or PBI desktop. Very strange.

[GilbertQ]

Hi there

From the spike in Memory it would indicate that there is a column of data that takes some time to compress.

When you look in the Task Manager which process is taking all the memory?