Forum Discussion

Georma's avatar
Georma
New Member
1 month ago
Solved

Why does PBI Service show different Semantic Model storage sizes when VertiPaq memory size is

I'm trying to understand a difference in Semantic Model storage reported by Power BI Service.


I have two semantic models:
*Both contain approximately 61 million rows.
*Both have almost the same schema and business logic.
*Both are published to the same Premium capacity, same region, and both use Large semantic model storage format.
*Both refresh successfully.


I analyzed both models using DAX Studio (VertiPaq Analyzer), and the results are almost identical:
Model A: 3.64 GB Total Size (Memory)
Model B: 3.65 GB Total Size (Memory)
So, from the VertiPaq perspective, the models consume essentially the same amount of memory.


However, in Workspace → Settings → System Storage, the reported storage is very different:
Model A: approximately 1.7 GB
Model B: approximately 3.6 GB


The main difference between the two models is how they are built:
Model A (Redshift): Data is imported from a base table, and most transformations and derived tables are created in Power Query.
Model B (ClickHouse): Most transformations are already performed in SQL, and Power BI imports the resulting tables directly.


I have already ruled out several possible causes:
Same VertiPaq memory size.
Same Premium capacity.
Same region.
Same Large semantic model storage format.
Re-publishing the model produces the same storage size.


My questions are:
What exactly does System Storage measure for a Semantic Model?


Besides the VertiPaq model, what other components are included in this storage calculation?


Can differences in model metadata, partitions, M queries, or the way the model is built affect System Storage, even when the VertiPaq memory size is the same?

 

Has anyone experienced a similar discrepancy?


Any insights would be greatly appreciated. Thanks!

  • Hi Georma  from what I have seen

    You can inspect things like partitions, model metadata, and object definitions using tabular Editor, but i dont thi k anything takes you perfectly to the system vakue storage. I think microsoft ticket will help with this. Do keep us posted

5 Replies

  • Hi Georma 

     System Storage is not the same thing as VertiPaq memory.

    What System Storage is measuring

    In Power BI Service, System Storage is the amount of workspace storage consumed by the semantic model item in the service. It is a service-side storage/accounting number, not the VertiPaq Analyzer number you see in DAX Studio.

    So when you see:

    • VertiPaq size ~3.65 GB in both models
    • but System Storage ~1.7 GB vs ~3.6 GB

    that means Power BI Service is counting more than just the compressed column store for one model, or it is accounting for the model differently in the service.

    What can be included besides VertiPaq

    Microsoft doesn’t publicly give a super-detailed byte-for-byte breakdown, but in practice the storage footprint can include things like:

    • Model metadata
      • tables, columns, relationships
      • measures, calculation groups
      • hierarchies, perspectives, translations
      • annotations, display folders, formatting, roles
    • Partitions
      • especially if the model has many partitions
      • incremental refresh can create multiple partitions
    • Power Query / M metadata
      • query definitions are stored with the model
      • source/lineage metadata may differ by design
    • Refresh policy / incremental refresh configuration
    • Service-managed item overhead
      • internal metadata the service keeps for the semantic model
      • possible versioning/snapshot-related overhead around refresh and publish operations

    Can model-building differences affect System Storage?

    Yes — indirectly.
    Even if VertiPaq memory is the same, the way the model is built can still affect service storage through:

    • number of partitions
    • incremental refresh setup
    • query and refresh metadata
    • model object complexity
    • service-generated internal metadata

    That said, if the VertiPaq footprint is essentially identical, then a 2x difference in System Storage is usually not explained by M vs SQL transformations alone. It more often points to one of these:

    1. Different partition structures
    2. Different incremental refresh or refresh history behavior
    3. Different service-side metadata overhead
    4. A storage accounting quirk or caching/compaction artifact in the service
    5. Different hidden model objects such as roles, translations, relationships, calc groups, or annotations

    Practical interpretation for your case

    Given your description, the most likely conclusion is:

    • VertiPaq Analyzer is telling the truth about model data size
    • System Storage is reflecting the total service storage footprint
    • the difference is probably due to non-data overhead, not the main fact table or business logic itself

    Why the Redshift/Power Query model might look smaller in System Storage

    One plausible explanation is that the model built more in Power Query may have:

    • fewer or simpler service-managed partitions
    • different refresh metadata
    • less internal overhead from how the model was authored or published

    Meanwhile, the SQL-first model may have:

    • more partitions
    • more complex source metadata
    • more service-side bookkeeping

    Bottom line

    • System Storage ≠ VertiPaq memory
    • It measures the semantic model’s storage footprint in the service, including more than the in-memory column store
    • Yes, metadata, partitions, M queries, refresh policies, and service-generated overhead can all contribute
    • If two models have the same VertiPaq size but different System Storage, the difference is usually in service overhead / metadata / partitioning, not the raw compressed data
    • Georma's avatar
      Georma
      New Member

      Thank you for the explanation. I understand the difference between VertiPaq memory and System Storage.

      However, my main question is what is actually consuming the additional 2 GB of System Storage.

      I have already verified that:

      • Both models have virtually the same VertiPaq size (3.64 GB vs. 3.65 GB).
      • Both are in the same Premium capacity.
      • Both use the Large semantic model storage format.
      • Both are hosted in the same region.
      • Republishing the model results in the same System Storage size.

      Is there any way to break down the System Storage footprint to understand what is consuming the additional space?

      For example, is there any tool, API, or administrative view that exposes how the Semantic Model storage is distributed (partitions, metadata, snapshots, service-managed objects, etc.)?

      I would like to identify the specific component responsible for the storage difference rather than just knowing that System Storage includes more than the VertiPaq model.

      • MohamedFowzan1's avatar
        MohamedFowzan1
        Super User

        Hi Georma  from what I have seen

        You can inspect things like partitions, model metadata, and object definitions using tabular Editor, but i dont thi k anything takes you perfectly to the system vakue storage. I think microsoft ticket will help with this. Do keep us posted

  • Hi Georma 

    This discrepancy is expected and comes down to what "System Storage" in Power BI Service actually measures vs what VertiPaq Analyzer shows.

     

    Short Answer:
    VertiPaq Analyzer = Only compressed data in memory
    System Storage in Service = VertiPaq data + Everything else in the .pbix/.abf file

     

    What System Storage includes:
    1. VertiPaq data - Your 3.64GB / 3.65GB. This matches DAX Studio.
    2. Model Metadata - Tables, columns, relationships, measures, roles. Complex models with lots of measures increase this.
    3.  L*Power Query M code + Refresh History  - This is the big one for you.
    -  Model A (Redshift + PQ transforms : All your Power Query steps, applied steps, and lineage are stored in the model. PQ stores intermediate results and query folding metadata. This can easily add 1-2GB+.
    -  Model B (ClickHouse + SQL transforms) : Since transforms are done in SQL, PQ has almost nothing to store. Just "Import from SQL". So metadata is tiny.
    4.  Partitions + Refresh Logs - Large models keep partition info and last few refresh logs in storage.
    5. Other objects - Images, themes, report pages embedded in the model if published as .pbix

     

    # Why Model A is ~1.7GB and Model B is ~3.6GB
    This is actually reversed from what I'd expect, but here's why it happens:
    - If Model A uses more query folding, Power BI can discard some PQ metadata after refresh and compress it better in the .abf file.
    - If Model B has many imported tables from SQL but with different data types/collations, the storage engine may store extra dictionaries.
    - Also check if Model B has "Store refresh history" or more partitions enabled. Each partition adds overhead.

    The key point:  VertiPaq size will always be similar because data is the same. System Storage will differ based on how the model was built.

     

    # How to reduce Model A's System Storage:
    1. In Power Query: Right-click queries > `Properties` > Uncheck "Enable load" for staging/derived tables you don't need in model
    2. `File > Options > Current File > Data Load` > Disable "Include in report refresh" for intermediate queries
    3. Reduce number of applied steps. Combine steps where possible.

    Yes, I and many others have seen 2x difference in System Storage with same VertiPaq size. It's almost always Power Query metadata.

     

    Hope this helps!