Trying to understand refresh memory usage

[MarioTechera]

Hi,

I have a situation that I hope someone can help me understand.

I'm running a semantic model in import mode.
The size of the model on disk is 1,5 GB.
When the model is opened in the PBI desktop it requires 2,76 GB memory according to the VertiPaq Analyzer.

Measure killer report the same amount of memory.

The model also runs in a F32 capacity premium workspace.

When I refresh the model locally on my PC I can see that SSAS requires between 8 and 9 GB memory by using the
task manager.

When I run a full refresh on the service I get the followin error message:
Refresh completed with warnings.
Data source error: Resource Governance: This operation was canceled because there wasn't enough memory to finish running it.
Either reduce the memory footprint of your dataset by doing things such as limiting the amount of imported data,
or if using Power BI Premium, increase the memory of the Premium capacity where this dataset is hosted.
More details: consumed memory 8413 MB, memory limit 8253 MB, database size before command execution 1986 MB.

Please note: I'm not looking for tips on reducing the model memory size.

I am looking for some transparency on understanding the numbers regarding memory consumption during refresh.

1) When my capacity is F32, the memory limit should be 10 GB per model.

The default limit is not being changed in the admin settings.
Why is there an error message before the 10 GB are being reached?
Nobody is querying the model while it is being processed.

2) The required memory for a full refresh expands to between 8 and 9 GB.

Why does this happen at all?
Is there any way to monitor this exactly during the processing?

I would usually assume that when a model is less than 3 GB according to the VertiPaq Analyzer as

well as Measure Killer, that a 10 GB, F32 Premium Workspace should suffice for a full refresh. Yet it doesn't.

We are working on reducing the footprint of the semantic model, but I would still like like to have transparency

on the current situation.

Any insight would be much appreciated!

Thanks and regards, Mario

[GilbertQ]

Hi @MarioTechera 

I would highly recommend using the Vertipaq analyzer to see if you have some columns that you don't need in your semantic model that you can reduce the size which will then affect the refresh sars and allow it to refresh successfully.

[MarioTechera]

Sorry, that wasn't what I asked.

[Shai_Karmani · Accepted solution]

The numbers actually fit a known pattern. The F32 capacity has a 10 GB cap per semantic model, which the engine treats as 10,240 MB. While a refresh is running, AS keeps the current copy of the model in memory so queries are not blocked, then builds the new copy beside it. The space available to the refresh is therefore the cap minus the current database size:

10,240 MB - 1,986 MB = 8,254 MB

That closely matches the "memory limit 8253 MB" you are seeing in the error.

For the 8 to 9 GB peak during refresh, that is normal. AS has to read source data, build dictionaries, sort columns, and produce the compressed column store, all of which run with intermediate structures that are much larger than the final compressed size. A working set of roughly 2x to 3x the final model is typical, so a model that lands at 2.76 GB can spike to 8 to 9 GB while being built.

You can watch it live from a tool like DAX Studio while the refresh runs:

SELECT * FROM $SYSTEM.DISCOVER_MEMORYUSAGE

In the service, the Fabric Capacity Metrics app exposes the same refresh memory events per operation.

If this helped, a thumbs up and accepting the solution would be appreciated.

Best regards,
Shai Karmani

Let's connect in LinkedIn