Forum Discussion
Significance testing
Microsoft EmployeeHi guyscottsouth,
There are many build-in statistical functions in Power BI. It seems this isn't a hard work for Power BI. If you can post more details, maybe I can translate them into DAX formula, which is a functional language used by Power BI.
Please reference these functions: https://msdn.microsoft.com/en-us/library/ee634822.aspx
and this post: significance-testing
Best Regards!
Dale
I have a much larger dataset, but this is a smaller version. I want to look by question at the result score for those answering yes against those answering no, in the full dataset the variances in the two samples are similar, but the sample size is quite variable. I was hopeing to use a 2 tailed T test to test the null hypothesis that the means of the two samples are the same. Ideally i would show the differences between the means and the confidence level.
| Assessment number | Question number | Answer | Overall Result |
| 1 | 1 | NO | 24.2 |
| 1 | 2 | YES | 24.2 |
| 1 | 3 | YES | 24.2 |
| 1 | 4 | NO | 24.2 |
| 1 | 5 | YES | 24.2 |
| 2 | 1 | YES | 68.9 |
| 2 | 2 | YES | 68.9 |
| 2 | 3 | NO | 68.9 |
| 2 | 4 | YES | 68.9 |
| 2 | 5 | YES | 68.9 |
| 3 | 1 | YES | 56.1 |
| 3 | 2 | NO | 56.1 |
| 3 | 3 | NO | 56.1 |
| 3 | 4 | YES | 56.1 |
| 3 | 5 | NO | 56.1 |
| 4 | 1 | NO | 74.2 |
| 4 | 2 | NO | 74.2 |
| 4 | 3 | NO | 74.2 |
| 4 | 4 | NO | 74.2 |
| 4 | 5 | NO | 74.2 |
| 5 | 1 | NO | 66.7 |
| 5 | 2 | YES | 66.7 |
| 5 | 3 | YES | 66.7 |
| 5 | 4 | YES | 66.7 |
| 5 | 5 | YES | 66.7 |
| 6 | 1 | NO | 11.6 |
| 6 | 2 | NO | 11.6 |
| 6 | 3 | YES | 11.6 |
| 6 | 4 | NO | 11.6 |
| 6 | 5 | YES | 11.6 |
| 7 | 1 | YES | 92.6 |
| 7 | 2 | YES | 92.6 |
| 7 | 3 | YES | 92.6 |
| 7 | 4 | YES | 92.6 |
| 7 | 5 | NO | 92.6 |
| 8 | 1 | NO | 48.8 |
| 8 | 2 | NO | 48.8 |
| 8 | 3 | NO | 48.8 |
| 8 | 4 | YES | 48.8 |
| 8 | 5 | YES | 48.8 |
| 9 | 1 | NO | 55.8 |
| 9 | 2 | NO | 55.8 |
| 9 | 3 | NO | 55.8 |
| 9 | 4 | YES | 55.8 |
| 9 | 5 | YES | 55.8 |
| 10 | 1 | NO | 94.3 |
| 10 | 2 | NO | 94.3 |
| 10 | 3 | YES | 94.3 |
| 10 | 4 | YES | 94.3 |
| 10 | 5 | YES | 94.3 |
| 11 | 1 | YES | 24.1 |
| 11 | 2 | NO | 24.1 |
| 11 | 3 | YES | 24.1 |
| 11 | 4 | NO | 24.1 |
| 11 | 5 | NO | 24.1 |
| 12 | 1 | NO | 16.0 |
| 12 | 2 | YES | 16.0 |
| 12 | 3 | YES | 16.0 |
| 12 | 4 | YES | 16.0 |
| 12 | 5 | NO | 16.0 |
| 13 | 1 | YES | 76.5 |
| 13 | 2 | NO | 76.5 |
| 13 | 3 | NO | 76.5 |
| 13 | 4 | YES | 76.5 |
| 13 | 5 | YES | 76.5 |
| 14 | 1 | NO | 32.8 |
| 14 | 2 | NO | 32.8 |
| 14 | 3 | NO | 32.8 |
| 14 | 4 | YES | 32.8 |
| 14 | 5 | NO | 32.8 |
| 15 | 1 | NO | 58.3 |
| 15 | 2 | NO | 58.3 |
| 15 | 3 | NO | 58.3 |
| 15 | 4 | YES | 58.3 |
| 15 | 5 | NO | 58.3 |
| 16 | 1 | NO | 37.2 |
| 16 | 2 | NO | 37.2 |
| 16 | 3 | NO | 37.2 |
| 16 | 4 | YES | 37.2 |
| 16 | 5 | NO | 37.2 |
Apologies for the bump, but i wondered if anyone had any further advice?
- v-jiascu-msft
Hi guyscottsouth,
I recalled my statistics. I don't know if this could help. Maybe you review it first, then we could go further. Please check this file:https://1drv.ms/u/s!ArTqPk2pu-BkgRFQZ5z_1PDA0QpQ.
Best Regards!
Dale
Thanks for the response, your time and effort are greatly appreciated, I'll have a look through tomorrow as you suggest and get back to you
regards
Guy
In your context, what does "overall result" indicate?
The overall result could be anything really, but it's probably most easily demonstrated as:
Q: Did you revise for your exam?
A: Yes
A: No
The result for each group would be the mean average score of the 'Yes' cohort and the mean average score for the 'No' ohort.
In this case, one might expect the the 'Yes' cohort average test score to be significantly different from the 'No' cohort, where as posing the question:
Q: Are you taller than 5'10"?
is somewhat less likely to show a statisticaly significant effect
I hope that helps
Thank you very much, that makes sense.
I know this is from a few years back, but did you end up using .pbix file Dale shared? We've been working to reverse engineer it a bit for our purposes but have hit a few snags.
Or, alternatively, have you created or encountered other DAX-based approaches for sig. testing in Power BI?