Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
filip1150
Frequent Visitor

Dax Measure question

Hi

 

I first posted this in the "DAX Commands and tips" but decided to copy it here as it seems mora appropriate,

 

I have a fact table based on data is already pre-aggregated at all levels. For privacy reasons I do not have access to the raw data.
Measures are things like count of people and median salary. Median salary is not really an additive measure and counts are neither (long explanation, is about some detailed data being hidden at very lower levels of granularity for privacy reasons).
Therefore I can't use any aggregate function to compute measures like averages, weighted averages, sums, they will have to be displayed the way they are stored.
My data will be a cross product of all dimensions members plus one "All" row for each dimension combination. Every dimension added to my data multiplies the number of rows by a factor of 1 (for all) plus the number of members in the dimension.
For instance, I will have dimensions like gender (Male/Female) and marital status (Married/Single), i will have the following rows

Male Single median_salary
Female Single median_salary
All genders Single median_salary
Male Married median_salary
Female Married median_salary
All genders Married median_salary
Male All Marital median_salary
Female All Marital median_salary
All genders All Marital median_salary

Data contains other dimensions with higher cardinality but I only have numbers available at each individual level, not for combinations. For instance if Marital status would also include "Divorced", I will get extra rows for "Divorced" but not metrics for combinations like "Divorced or Single", "Divorced or married" etc.
My challenge is to medel the data so that people can report on it. For instance if a report does not contain a "Gender" column I want to see the numbers (salary) corresponding to "All genders", but if data contains a Gender column or filter I want to show the numbers corresponding to the appropriate gender.
Also, if the data is filtered such as more members but not all are take, my dimension should be empty. (I.e. if you filter by multiple values and choose single and divorced then I want to see the measure empty as I don;t know the correct result)

I was thinking maybe the isFiltered function might help somehow, but I'm not quite sure, and my power BI / dax experience is limited.

 

Thanks!

1 ACCEPTED SOLUTION

Just to set expectations, I want to start out by saying that this concept raises a number of red flags for me. I think it might be possible, but I don't know if the performance will scale as you add more columns and more combinations.

 

You are basically taking one of the worlds fastest aggregation engines, then loading in pre-aggregated data and only using it for filtering. And a good deal of the speed of the tabular engine behind Power BI comes from the fact that it uses a column store, so if you have a visual that only references 4 columns, Power BI only has to scan those 4 columns. But with your non-aggregatable data it will have to scan every "row" of all columns for every visual.

 

Based on your requirements and the sample data you provided I think the following measure will work

 

Measure = 
// Marital Status Filters
VAR _maritalStatusFilter =  FILTER(VALUES(Table1[MaritalStatus]),  
    IF(ISINSCOPE(Table1[MaritalStatus]) 
        , Table1[MaritalStatus] <> "All Marital Statuses"                // if Marital Status is one of the output columns
        , Table1[MaritalStatus] = SELECTEDVALUE(Table1[MaritalStatus])   // if Marital Status has a single filter
    )       
)
VAR _maritalStatusAll = FILTER(ALL(Table1[MaritalStatus])  
    , IF(NOT(HASONEVALUE(Table1[MaritalStatus]))                         // if Marital Status has multiple fiters
        , Table1[MaritalStatus] = "All Marital Statuses"
        ,FALSE()
    )  
)
// Gender Filters
VAR _genderFilter =  FILTER(VALUES(Table1[Gender]),  
    IF(ISINSCOPE(Table1[Gender]) 
        , Table1[Gender] <> "All genders"
        , Table1[Gender] = SELECTEDVALUE(Table1[Gender])
    )
)
VAR _genderAll = FILTER(ALL(Table1[Gender])  
    , IF(NOT(HASONEVALUE(Table1[Gender]))
        , Table1[Gender] = "All genders"
        ,FALSE()
    )
)
// Year Filter (does not need an "All" filter)
VAR _yearFilter =  FILTER(VALUES(Table1[Year])  
    , Table1[Year] = SELECTEDVALUE(Table1[Year])
)
Var result = CALCULATE(MAX(Table1[m])
, UNION(_genderFilter,_genderAll)
, UNION( _maritalStatusFilter, _maritalStatusAll)
, _yearFilter
)
return result

View solution in original post

8 REPLIES 8
v-danhe-msft
Microsoft Employee
Microsoft Employee

Hi @filip1150 ,

Could you please post some simple sample data and your desired result to have a test if possible? Please see this post regarding How to Get Your Question Answered Quickly: https://community.powerbi.com/t5/Community-Blog/How-to-Get-Your-Question-Answered-Quickly/ba-p/38490

 

Regards,

Daniel He

Community Support Team _ Daniel He
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Good point, thank you.

Below is my data and here are the requirements:

 

If I create a table visual, I expect the following behavior:
1. If I do not filter/slice on year, the measure should be blank. I.e. creating a table without a "Year" column should result in all rows having an empty measure.
Also, if I filter /slice for multiple years the measures should be blank
2. If the table does not have a "Marital" column and there is no slicer/filter on Marital, I expect to show the values corresponding to the "All Marital Statuses rows. also, I cannot aggregate 2 differnt marital statuses on the same row - should be an empty measure.
Same goes for gender.
3. If my table contains a Year, Gender and Marital column, I do not want to see the rows that have Marital = "All Marital" or Gender"All Gender"
4. If the table contains only year and Measure and is not otherwise filtered by gender or Marital, I expect to see 3 rows (one for each year) corresponding to Gender= "All genders" AND Marital = "all marital" rows

The "All Gender" and "All Marital" rows act like subtotals and are included in the source data because their values cannot be inferred from lower level values.
The actual data contains about 50 columns and few hundred million rows.
If this is possible I might have a follow up question about hierarchies.

 


Year Gender MaritalStatus m
2017 Male Single 2017/Male/Single
2017 Male Married 2017/Male/Married
2017 Male Divorced 2017/Male/Divorced
2017 Male All Marital Statuses 2017/Male
2017 Female Single 2017/Female/Single
2017 Female Married 2017/Female/Married
2017 Female Divorced 2017/Female/Divorced
2017 Female All Marital Statuses 2017/Female
2017 All genders Single 2017/Single
2017 All genders Married 2017/Married
2017 All genders Divorced 2017/Divorced
2017 All genders All Marital Statuses 2017
2018 Male Single 2018/Male/Single
2018 Male Married 2018/Male/Married
2018 Male Divorced 2018/Male/Divorced
2018 Male All Marital Statuses 2018/Male
2018 Female Single 2018/Female/Single
2018 Female Married 2018/Female/Married
2018 Female Divorced 2018/Female/Divorced
2018 Female All Marital Statuses 2018/Female
2018 All genders Single 2018/Single
2018 All genders Married 2018/Married
2018 All genders Divorced 2018/Divorced
2018 All genders All Marital Statuses 2018
2019 Male Single 2019/Male/Single
2019 Male Married 2019/Male/Married
2019 Male Divorced 2019/Male/Divorced
2019 Male All Marital Statuses 2019/Male
2019 Female Single 2019/Female/Single
2019 Female Married 2019/Female/Married
2019 Female Divorced 2019/Female/Divorced
2019 Female All Marital Statuses 2019/Female
2019 All genders Single 2019/Single
2019 All genders Married 2019/Married
2019 All genders Divorced 2019/Divorced
2019 All genders All Marital Statuses 2019

Sorry,posting again the data.

"m" is my measure. I chose it to be string for this example to make it more readable and to emphasize is cannot be aggregated.

 

If I create a table visual, I expect the following behavior:
1. If I do not filter/slice on year, the measure should be blank. I.e. creating a table without a "Year" column should result in all rows having an empty measure.
Also, if I filter /slice for multiple years the measures should be blank
2. If the table does not have a "Marital" column and there is no slicer/filter on Marital, I expect to show the values corresponding to the "All Marital Statuses rows. also, I cannot aggregate 2 differnt marital statuses on the same row - should be an empty measure.
Same goes for gender.
3. If my table contains a Year, Gender and Marital column, I do not want to see the rows that have Marital = "All Marital" or Gender"All Gender"
4. If the table contains only year and Measure and is not otherwise filtered by gender or Marital, I expect to see 3 rows (one for each year) corresponding to Gender= "All genders" AND Marital = "all marital" rows

The "All Gender" and "All Marital" rows act like subtotals and are included in the source data because their values cannot be inferred from lower level values.
The actual data contains about 50 columns and few hundred million rows.
If this is possible I might have a follow up question about hierarchies.

 

YearGenderMaritalStatusm
2017MaleSingle2017/Male/Single
2017MaleMarried2017/Male/Married
2017MaleDivorced2017/Male/Divorced
2017MaleAll Marital Statuses2017/Male
2017FemaleSingle2017/Female/Single
2017FemaleMarried2017/Female/Married
2017FemaleDivorced2017/Female/Divorced
2017FemaleAll Marital Statuses2017/Female
2017All gendersSingle2017/Single
2017All gendersMarried2017/Married
2017All gendersDivorced2017/Divorced
2017All gendersAll Marital Statuses2017
2018MaleSingle2018/Male/Single
2018MaleMarried2018/Male/Married
2018MaleDivorced2018/Male/Divorced
2018MaleAll Marital Statuses2018/Male
2018FemaleSingle2018/Female/Single
2018FemaleMarried2018/Female/Married
2018FemaleDivorced2018/Female/Divorced
2018FemaleAll Marital Statuses2018/Female
2018All gendersSingle2018/Single
2018All gendersMarried2018/Married
2018All gendersDivorced2018/Divorced
2018All gendersAll Marital Statuses2018
2019MaleSingle2019/Male/Single
2019MaleMarried2019/Male/Married
2019MaleDivorced2019/Male/Divorced
2019MaleAll Marital Statuses2019/Male
2019FemaleSingle2019/Female/Single
2019FemaleMarried2019/Female/Married
2019FemaleDivorced2019/Female/Divorced
2019FemaleAll Marital Statuses2019/Female
2019All gendersSingle2019/Single
2019All gendersMarried2019/Married
2019All gendersDivorced2019/Divorced
2019All gendersAll Marital Statuses2019

Just to set expectations, I want to start out by saying that this concept raises a number of red flags for me. I think it might be possible, but I don't know if the performance will scale as you add more columns and more combinations.

 

You are basically taking one of the worlds fastest aggregation engines, then loading in pre-aggregated data and only using it for filtering. And a good deal of the speed of the tabular engine behind Power BI comes from the fact that it uses a column store, so if you have a visual that only references 4 columns, Power BI only has to scan those 4 columns. But with your non-aggregatable data it will have to scan every "row" of all columns for every visual.

 

Based on your requirements and the sample data you provided I think the following measure will work

 

Measure = 
// Marital Status Filters
VAR _maritalStatusFilter =  FILTER(VALUES(Table1[MaritalStatus]),  
    IF(ISINSCOPE(Table1[MaritalStatus]) 
        , Table1[MaritalStatus] <> "All Marital Statuses"                // if Marital Status is one of the output columns
        , Table1[MaritalStatus] = SELECTEDVALUE(Table1[MaritalStatus])   // if Marital Status has a single filter
    )       
)
VAR _maritalStatusAll = FILTER(ALL(Table1[MaritalStatus])  
    , IF(NOT(HASONEVALUE(Table1[MaritalStatus]))                         // if Marital Status has multiple fiters
        , Table1[MaritalStatus] = "All Marital Statuses"
        ,FALSE()
    )  
)
// Gender Filters
VAR _genderFilter =  FILTER(VALUES(Table1[Gender]),  
    IF(ISINSCOPE(Table1[Gender]) 
        , Table1[Gender] <> "All genders"
        , Table1[Gender] = SELECTEDVALUE(Table1[Gender])
    )
)
VAR _genderAll = FILTER(ALL(Table1[Gender])  
    , IF(NOT(HASONEVALUE(Table1[Gender]))
        , Table1[Gender] = "All genders"
        ,FALSE()
    )
)
// Year Filter (does not need an "All" filter)
VAR _yearFilter =  FILTER(VALUES(Table1[Year])  
    , Table1[Year] = SELECTEDVALUE(Table1[Year])
)
Var result = CALCULATE(MAX(Table1[m])
, UNION(_genderFilter,_genderAll)
, UNION( _maritalStatusFilter, _maritalStatusAll)
, _yearFilter
)
return result

@filip1150 How did you get on with this? Did it work for you? Was the performance acceptible over your full data set?

@d_gosbell  

Navigating data in power BI is rather sluggish but not totally un-acceptable at this point.

Creating a table visual that includes the measure will take about 15-20 seconds for every click where you add or remove columns. The file size is about 0.6 GB. The base SQL table consists of about 120 million rows, including about 10 dimensions and 10 measures. This running an a pretty old computer, slow hdd, 16g GB ram, i5 4 core@ 3.3GHZ

My next question at this point what is the best way to hide the aggregate rows (i.e. if i select gender, I do not want to see the "All Genders". And how to deal with hierarchies - i.e. I have data at each level of aggregations and they will not roll up nicely)


@filip1150 wrote:

My next question at this point what is the best way to hide the aggregate rows (i.e. if i select gender, I do not want to see the "All Genders". And how to deal with hierarchies - i.e. I have data at each level of aggregations and they will not roll up nicely)


Am I correct in assuming that you are talking about hiding these rows when the column is used in a slicer? If so there is no way at the moment that I know of to do this, but the Power BI team is working on a feature to allow for applying filters in slicers https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/12545673-visual-level-filtering-i... which should address this (if my guess about your requirement here is correct)

 


@filip1150 wrote:

And how to deal with hierarchies - i.e. I have data at each level of aggregations and they will not roll up nicely)


Can you provide a table of data like you did with the gender/marital status example? I'm not entirely sure what the issue is here.

Hi Darren

First of all, many thanks for your help.

I had to wait to get approval to install a newer version of Power BI (the one I had did not have the required DAX functions available).

I just got that this morning, so I will have to try and see what kind of performance I get. I will report back on that as soon as I have something, probably sometime next week

 

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

AugPowerBI_Carousel

Power BI Monthly Update - August 2024

Check out the August 2024 Power BI update to learn about new features.

September Hackathon Carousel

Microsoft Fabric & AI Learning Hackathon

Learn from experts, get hands-on experience, and win awesome prizes.

Sept NL Carousel

Fabric Community Update - September 2024

Find out what's new and trending in the Fabric Community.