Forum Discussion
Calculating Coefficient of Variation
Hello everyone ๐
I do have a table containing the following columns:
Personal Number, Date, Team, Project, TimeCategory, Time
There are multiple Personal Numbers per Team, multiple Teams per Project and also multiple TimeCategories per Date and Personal Number.
It is my goal to determine whether the high qutoas of illness are due to some extreme examples of high illness rates or a case of illness rates being high in general.
In order to answer this question, I want to use a statistical tool called the coefficient of variation. This is the standard deviation divided by the expected value.
https://en.wikipedia.org/wiki/Coefficient_of_variation
Because of different contract lengths, I will not use some equal distribution of the quotas.
I will use the illness quota per employee as a random variable with probability being equal to the proportion of the employees hours to the hours of the team/project within the selected time period.
Times relevant may be filtered using the column TimeCategory.
I know how to use CALUCATE() im combination with FILTER(), ALL(), ALLEXCEPT(), but I am lost here.
My problem is: The data must be dynamically preaggregated by Personal Number in order to get the right results. And I do not not know how to do this.
The formula itself will look like this:
CV = (((Illness Quota of employee A- Illness Quota of Team)^2)*(Total Hours of Employee A)/(Total Hours of Team) +.....+((Illness Quota of employee Y- Illness Quota of Team)^2)*(Total Hours of Employee Y)/(Total Hours of Team) )/(IllnessQuota of Team)
As said: My problem is the fact, that the data are not preaggregated by Personal Number.
Need help!
- Anonymous6 years ago
All filtering is one-way only from a dimension to the fact table.
// Dimensions: // Employee connected to FactTable[PIN] (1:*) // Calendar connected to FactTable[Date] (1:*) // Team connected to FactTable[TeamId] (1:*) // Project connected to FactTable[ProjectId] (1:*) // TimeCategory connected to FactTable[TimeCategoryId] (1:*) // All *Id fields are hidden in dimensions. // // All columns in the FactTable must be hidden. // Only measures can be visible. All slicing is // done through dimensions. This is the correct // star schema model. // Define the following measures. They work for any // slice for any dimension. In particular for // slices on Employee. [Total] = SUM( FactTable[Hours] ) [Illness] = CALCULATE( [Total], KEEPFILTERS( TimeCategory[TimeCategoryId] = "Illness" ) ) [IllnessQuota] = DIVIDE( [Illness], [Total] ) [Proportion To Team] = var __totalForEmps = [Total] var __teamsOfEmps = summarize( FactTable, Team[TeamId] ) var __totalForTeams = calculate( [Total], __teamsOfEmps, all( Employee ), all( Team ) ) var __result = divide( __totalForEmps, __totalForTeams ) return __result [VC for Team] = var __oneTeamVisible = hasonevalue( Team[TeamId] ) var __team = SUMMARIZE( FactTable, Team[TeamId] ) var __employees = SUMMARIZE( FactTable, Employee[PIN] ) var __numerator = SQRT( SUMX( __employees, var __iqForEmp = [IllnessQuota] var __iqForTeam = calculate( [IllnessQuota], __team, all( Team ), all( Employee ) ) var __propToTeam = [Proportion To Team] var __result = __propToTeam * POWER( __iqForEmp - __iqForTeam, 2 ) return __result ) ) var __denominator = calculate( [IllnessQuota], __team, all( Team ), all( Employee ) ) var __varCoeff = DIVIDE( __numerator, __denominator ) return if( __oneTeamVisible, __varCoeff )Once you've implemented the correct model, please let me know how it goes. Thanks ๐
Best
D
10 Replies
- AnonymousNot applicableHi there.
Mate, would you please make this more understandable? Please bear in mind that we are not working with your model and what's easily graspable to you is not necessarily easy to grasp for us. You are referring in your formulas to fields that are nowhere to be found in your description. The table at the top you are mentioning does not give us almost any information at all. For instance, where and what is 'illness quota of employee'? I understand there are dimensions and some fact tables in your model but could you be more explicit, please? State what tables you have, what measures, what columns... and how they are linked together.
If ppl don't understand your problem, you won't get any replies.
Best
D- Schmidtmayer
Helper II
First of all, thanks for the reply.
I will be more specific.
There is only one table til now, having the mentioned columns:
Personal Number - Unique ID to identify the employees
Date - no further infos required, I suppose
Team - the current team of the employee
Project -the current project of the employee
TimeCategory - Classification of hours, includes the categories Illness, Productive, Vacation
Hours - Registered time of the employee
In the following let X be the set of Personal Numbers, T be the set of Teams
For x from X we define:
Total(x) = CALCULATE(SUM(Hours); Personal Number = x)
Illness(x) = CALCULATE(SUM(Hours); Personal Number =x; TimeCategory = Illness)
For t from T we define:
Total(t) = CALCULATE(SUM(Hours); Team = t)
Illness(t) = CALCULATE(SUM(Hours); Team = t; TimeCategory = Illness)
Now, the quotas:
IllnessQuota(x) = Illness(x)/Total(x)
IllnessQuota(t) = Illness(t)/Total(t)
Lastly:
Proportion(x) = Total(x)/Total(t)
Remark: Every employee is a member of just one team.
Then I wish to calculate
VC(t) =
SQRT(
SUM(
(IllnessQuota(x) - IllnessQuota(t))^2รProportion(x)
)
)
/IllnessQuota(t)
Just these x shall be included, which where a part of t at this time.
A table with the following is also present:
Personal Number, Team, Start
Giving the complete history of teams. The current team of x is mentioned in the first table, so I think this one should not be needed.
CV(t) should be visualized in a bar diagramm having years on the x axis, with Drill Down to month and having a filter for teams.- AnonymousNot applicable
All filtering is one-way only from a dimension to the fact table.
// Dimensions: // Employee connected to FactTable[PIN] (1:*) // Calendar connected to FactTable[Date] (1:*) // Team connected to FactTable[TeamId] (1:*) // Project connected to FactTable[ProjectId] (1:*) // TimeCategory connected to FactTable[TimeCategoryId] (1:*) // All *Id fields are hidden in dimensions. // // All columns in the FactTable must be hidden. // Only measures can be visible. All slicing is // done through dimensions. This is the correct // star schema model. // Define the following measures. They work for any // slice for any dimension. In particular for // slices on Employee. [Total] = SUM( FactTable[Hours] ) [Illness] = CALCULATE( [Total], KEEPFILTERS( TimeCategory[TimeCategoryId] = "Illness" ) ) [IllnessQuota] = DIVIDE( [Illness], [Total] ) [Proportion To Team] = var __totalForEmps = [Total] var __teamsOfEmps = summarize( FactTable, Team[TeamId] ) var __totalForTeams = calculate( [Total], __teamsOfEmps, all( Employee ), all( Team ) ) var __result = divide( __totalForEmps, __totalForTeams ) return __result [VC for Team] = var __oneTeamVisible = hasonevalue( Team[TeamId] ) var __team = SUMMARIZE( FactTable, Team[TeamId] ) var __employees = SUMMARIZE( FactTable, Employee[PIN] ) var __numerator = SQRT( SUMX( __employees, var __iqForEmp = [IllnessQuota] var __iqForTeam = calculate( [IllnessQuota], __team, all( Team ), all( Employee ) ) var __propToTeam = [Proportion To Team] var __result = __propToTeam * POWER( __iqForEmp - __iqForTeam, 2 ) return __result ) ) var __denominator = calculate( [IllnessQuota], __team, all( Team ), all( Employee ) ) var __varCoeff = DIVIDE( __numerator, __denominator ) return if( __oneTeamVisible, __varCoeff )Once you've implemented the correct model, please let me know how it goes. Thanks ๐
Best
D
- Greg_Deckler
Community Champion
I'll have to look, I believe I cover Covariance in my upcoming book, DAX Cookbook. Comes out next week. But would need sample data. Please see this post regarding How to Get Your Question Answered Quickly: https://community.powerbi.com/t5/Community-Blog/How-to-Get-Your-Question-Answered-Quickly/ba-p/38490
- Greg_Deckler
Community Champion
Ah yes, here is the formula that I was using for covariance.
Covariance = VAR __Table = 'R04_Table' VAR __Count = COUNTROWS(__Table) VAR __AvgA = AVERAGEX(__Table,[A]) VAR __AvgB = AVERAGEX(__Table,[B]) VAR __Table1 = ADDCOLUMNS( __Table, "__Covariance", DIVIDE( ([A] - __AvgA) * ([B] - __AvgB), __Count ) ) RETURN SUMX(__Table1,[__Covariance])