Vertipaq engine bug - chooses hash instead of valu...

Anonymous · ‎11-19-2020

Have a dataset that has 55M rows.
For a fact column that is used for aggregations and sums that has a high cardinality Vertipaq engine was storing it as HASH value, adding a 1.2GB dictionary to the model and accounting for 50% of the size. If the dataset was built for a subset (i.e. one month) then the engine choose Value.

By changing the type to INT64 the engine has now choosen Value encoding. Below is a screen grab of the hash\value effects.

Value is what I would expect and it performs a lot better on the service

Can this be logged as a bug by Power BI team and a fix done - or provide the ability to override and define the encoding type?
I also tried using the hint as mentioned in another post but this didn't change the encoding type.

Anonymous · ‎11-23-2020

Hi @v-stephen-msft
Thanks so much for your response - I hadn't seen some of those responses - after reading them they all provide similar advice in that a hint can be applied but it may not fix the issue - and most respondants report that the hint didn't work.

I have tried the hint with no success and as mentioned by others there is no guarantee that a hint will force the Veritpaq engine in choosing the encoding strategy to use for the column.

Given the other number of affected users maybe this should be raised as either a bug request to tweak the logic for the vertipaq engine or provide a new feature to be able to force\define the encoding type for these sorts of scenarios.