Not enough memory error for powerbi data refresh on fabric workspace

[jigneshpatel]

Hi
We have been using power bi pro licenses and we gets various data tables from Azure SQL server to powerbi. Total data size of all data being synched to powerbi is 940 MB

My company has decided to shift to Fabric and purchased F2 capacity from Azure to start with. We have changed the workspace license from pro to fabric and selected our F2 capacity. After that the scheduled synchronization gets failed with following error.

Resource Governing: This operation was canceled because there wasn't enough memory to finish running it. Either reduce the memory footprint of your dataset by doing things such as limiting the amount of imported data, or if using Power BI Premium, increase the memory of the Premium capacity where this dataset is hosted. More details: consumed memory 426 MB, memory limit 425 MB, database size before command execution 2646 MB. See https://go.microsoft.com/fwlink/?linkid=2159753 to learn more.
I am confused here as the F2, F4 and F8 all has 3 GB memory. Total data being synched is less than one GB. If we consider caching then also it should not go beyond 2 GB.

Why we are getting this error.

Can anyone help me understanding the scene behind this issue? And guide me how to migrate to fabric?

Jignesh

[Anonymous]

Thanks for the replies from pallavi_r and Arun_kumar.

Hi @jigneshpatel ,

Have you solved your problem? If it is solved, please share your solution and accept it as solution or mark the helpful replies, it will be helpful for other members of the community who have similar problems as yours to solve it faster. Thank you very much for your kind cooperation!

Best Regards,
Zhu

[pallavi_r]

Hi @jigneshpatel ,

When you say your dataset size is 940 mb but the error message says its 2646 mb, that means 940 gb is approx dataset size but during the refresh process memory size might have got doubled due to calculated columns or tables. Each time we refresh a failed dataset, the memory gets further increased. So we will have to republish and try refresh again to see if its really the calculated column/table issues or there is some other backgroubd process that coincided with refresh.

We can check in the fabric capacity metrics app if it has reached throttling or over utilization during the first full refresh.

Thanks,

Pallavi

[Arun_kumar]

Consider optimizing the calculated columns and tables. Instead of using Power BI for calculations, you can create those calculated columns at the server level using an SQL query.

[Arun_kumar]

You can also reduce the data size by filtering it to the last 2 years if you have historical data beyond that. Remember, calculated columns or tables process all rows in the table, which can consume more memory.

[Arun_kumar]

I used a similar approach in the past when I faced the same issue.

[Anonymous]

Hi @jigneshpatel ,

Based on the error message you provided, it appears that your semantic model has already consumed 2646 MB at the start of the command. so when a command operation attempts to consume more memory than the limit allows, it causes the operation to fail.

Although F2 capacity provides 3 GB of memory, memory is not limited to the size of the data, but also includes memory required for query execution, caching, and other background processes.

You can reduce the memory footprint of your semantic model by limiting the amount of data loaded with each refresh.
And you can use the app to monitor your capacity consumption and make informed decisions on how to use your capacity resources: Install the Microsoft Fabric capacity metrics app - Microsoft Fabric | Microsoft Learn

Best Regards,
Zhu

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.