Data compression Procedure

Hi pennyhoho117 

1. When an import mode report is published to Power BI Service, how is the data being compressed and optimized?

When a report in Import Mode is published to Power BI Service, the underlying data model (powered by VertiPaq) is highly compressed and optimized using columnar storage and dictionary encoding. The key optimizations include:

• Columnar Storage: Data is stored in a column-wise manner, reducing redundancy.
• Encoding & Dictionary Compression: Repetitive values are stored as dictionary references rather than full values.
• Run-Length Encoding (RLE): Sequences of repeated values are replaced with a single value and a count.
• Bit-Packing & Data Type Optimization: Efficient storage of numeric values using the smallest required data types.
  This compression and optimization happen before the report is published to Power BI Service.

2. After it has been compressed and optimized, does VertiPaq achieve a 10x compression?

Yes, typically VertiPaq can achieve 10x compression, but the actual ratio depends on:

• The structure of your data (number of unique values, data types, etc.).
• The presence of high cardinality columns (e.g., unique IDs reduce compression efficiency).
• How efficiently dictionary encoding and RLE can be applied.
  Some datasets may achieve higher compression (20x-100x), while others with many unique values may see less.

3. Is the complete process: Publish report > Compressed and optimized > Achieve 10x compression by VertiPaq?

Not exactly, the correct sequence is:

1. Data Import & Model Creation: Data is loaded into Power BI Desktop.
2. Compression & Optimization (VertiPaq Engine): Data is compressed and optimized within Power BI Desktop.
3. Publish to Power BI Service: The optimized .pbix file (including the compressed model) is uploaded to Power BI Service.
4. Storage in Power BI Service: The compressed model is stored in memory when loaded.

4. How does a published report refresh in Power BI Service? Does the Data Gateway compress and optimize data?

When a report is already published to Power BI Service and a Scheduled Refresh runs:

1. Data Source Query Execution: Power BI queries the source (via a Data Gateway if the source is on-premises).
2. Data Compression (via VertiPaq) in Power BI Service:
   • The refreshed data is re-compressed and optimized using the same VertiPaq techniques.
3. Storage in Power BI Service: The newly compressed data replaces the old dataset in Power BI Service.

🔹 The Data Gateway does not perform compression—it only facilitates query execution and data transfer. The actual compression happens inside Power BI Service, just like it does in Power BI Desktop.