Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get certified in Microsoft Fabric—for free! For a limited time, the Microsoft Fabric Community team will be offering free DP-600 exam vouchers. Prepare now

Reply
Anonymous
Not applicable

Vertipaq engine bug - chooses hash instead of value for high cardinality fact column

Have a dataset that has 55M rows. 
For a fact column that is used for aggregations and sums that has a high cardinality Vertipaq engine was storing it as HASH value, adding a 1.2GB dictionary to the model and accounting for 50% of the size. If the dataset was built for a subset (i.e. one month) then the engine choose Value. 

By changing the type to INT64 the engine has now choosen Value encoding. Below is a screen grab of the hash\value effects.

Value is what I would expect and it performs a lot better on the service

Can this be logged as a bug by Power BI team and a fix done - or provide the ability to override and define the encoding type?
I also tried using the hint as mentioned in another post but this didn't change the encoding type.

power-bi-ftw_0-1605852956110.png

 

2 REPLIES 2
Anonymous
Not applicable

Hi @v-stephen-msft 
Thanks so much for your response - I hadn't seen some of those responses - after reading them they all provide similar advice in that a hint can be applied but it may not fix the issue - and most respondants report that the hint didn't work.

I have tried the hint with no success and as mentioned by others there is no guarantee that a hint will force the Veritpaq engine in choosing the encoding strategy to use for the column.

Given the other number of affected users maybe this should be raised as either a bug request to tweak the logic for the vertipaq engine or provide a new feature to be able to force\define the encoding type for these sorts of scenarios.

v-stephen-msft
Community Support
Community Support

Hi @Anonymous ,

 

Sorry for my late reply.

 

I found these posts which may be helpful to solve your problem:

 

https://community.powerbi.com/t5/Desktop/VertiPaq-decimal-columns-encoding-type/td-p/959317

 

https://community.powerbi.com/t5/Desktop/Vertipaq-Column-encoding-HASH-v-VALUE/td-p/904256

 

https://community.powerbi.com/t5/Desktop/Vertipaq-Engine-VALUE-vs-HASH/m-p/690874#M333145

 

 

 

Best Regards,

Stephen Tao

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
OCT PBI Update Carousel

Power BI Monthly Update - October 2024

Check out the October 2024 Power BI update to learn about new features.

September Hackathon Carousel

Microsoft Fabric & AI Learning Hackathon

Learn from experts, get hands-on experience, and win awesome prizes.

October NL Carousel

Fabric Community Update - October 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors