cancel
Showing results for
Did you mean:

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Frequent Visitor

## Significance testing

I have two unequally sized samples with similar varience and broadly normal distributions. What is the best way to perform a two tailed significance test to determine if the two means are significantly difference nd how do I do this in power BI desktop?

Can a 2 tailed t test be done easily?

11 REPLIES 11
Employee

There are many build-in statistical functions in Power BI. It seems this isn't a hard work for Power BI. If you can post more details, maybe I can translate them into DAX formula, which is a functional language used by Power BI.

and this post: significance-testing

Best Regards!

Dale

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Frequent Visitor

I have a much larger dataset, but this is a smaller version. I want to look by question at the result score for those answering yes against those answering no, in the full dataset the variances in the two samples are similar, but the sample size is quite variable. I was hopeing to use a 2 tailed T test to test the null hypothesis that the means of the two samples are the same. Ideally i would show the differences between the means and the confidence level.

 Assessment number Question number Answer Overall Result 1 1 NO 24.2 1 2 YES 24.2 1 3 YES 24.2 1 4 NO 24.2 1 5 YES 24.2 2 1 YES 68.9 2 2 YES 68.9 2 3 NO 68.9 2 4 YES 68.9 2 5 YES 68.9 3 1 YES 56.1 3 2 NO 56.1 3 3 NO 56.1 3 4 YES 56.1 3 5 NO 56.1 4 1 NO 74.2 4 2 NO 74.2 4 3 NO 74.2 4 4 NO 74.2 4 5 NO 74.2 5 1 NO 66.7 5 2 YES 66.7 5 3 YES 66.7 5 4 YES 66.7 5 5 YES 66.7 6 1 NO 11.6 6 2 NO 11.6 6 3 YES 11.6 6 4 NO 11.6 6 5 YES 11.6 7 1 YES 92.6 7 2 YES 92.6 7 3 YES 92.6 7 4 YES 92.6 7 5 NO 92.6 8 1 NO 48.8 8 2 NO 48.8 8 3 NO 48.8 8 4 YES 48.8 8 5 YES 48.8 9 1 NO 55.8 9 2 NO 55.8 9 3 NO 55.8 9 4 YES 55.8 9 5 YES 55.8 10 1 NO 94.3 10 2 NO 94.3 10 3 YES 94.3 10 4 YES 94.3 10 5 YES 94.3 11 1 YES 24.1 11 2 NO 24.1 11 3 YES 24.1 11 4 NO 24.1 11 5 NO 24.1 12 1 NO 16.0 12 2 YES 16.0 12 3 YES 16.0 12 4 YES 16.0 12 5 NO 16.0 13 1 YES 76.5 13 2 NO 76.5 13 3 NO 76.5 13 4 YES 76.5 13 5 YES 76.5 14 1 NO 32.8 14 2 NO 32.8 14 3 NO 32.8 14 4 YES 32.8 14 5 NO 32.8 15 1 NO 58.3 15 2 NO 58.3 15 3 NO 58.3 15 4 YES 58.3 15 5 NO 58.3 16 1 NO 37.2 16 2 NO 37.2 16 3 NO 37.2 16 4 YES 37.2 16 5 NO 37.2

Anonymous
Not applicable

In your context, what does "overall result" indicate?

Frequent Visitor

The overall result could be anything really, but it's probably most easily demonstrated as:

Q: Did you revise for your exam?

A: Yes

A: No

The result for each group would be the mean average score of the 'Yes' cohort and the mean average score for the 'No' ohort.

In this case, one might expect the the 'Yes' cohort average test score to be significantly different from the 'No' cohort, where as posing the question:

Q: Are you taller than 5'10"?

is somewhat less likely to show a statisticaly significant effect

I hope that helps

Anonymous
Not applicable

Thank you very much, that makes sense.

I know this is from a few years back, but did you end up using .pbix file Dale shared? We've been working to reverse engineer it a bit for our purposes but have hit a few snags.

Or, alternatively, have you created or encountered other DAX-based approaches for sig. testing in Power BI?

Frequent Visitor

It was helpul, but I ended up going back and building something from first principles. What I would say is that the exercise is significantly easier in R or python, so I've subsequently used that

Frequent Visitor

Apologies for the bump, but i wondered if anyone had any further advice?

Employee

I recalled my statistics. I don't know if this could help. Maybe you review it first, then we could go further. Please check this file:https://1drv.ms/u/s!ArTqPk2pu-BkgRFQZ5z_1PDA0QpQ.

Best Regards!

Dale

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
New Member

Hi Dale, do you stil have the file? I am Looking for the exact same thing. The link is not valid anymore.

Thanks

Anonymous
Not applicable

This is awesome Dale. Are you by chance available at all to work me though how I might re-create this to generate a template table I can use on all or most of my data sets?

Frequent Visitor

Thanks for the response, your time and effort are greatly appreciated, I'll have a look through tomorrow as you suggest and get back to you

regards

Guy

Announcements

#### Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

#### Power BI Monthly Update - June 2024

Check out the June 2024 Power BI update to learn about new features.

#### Fabric Community Update - June 2024

Get the latest Fabric updates from Build 2024, key Skills Challenge voucher deadlines, top blogs, forum posts, and product ideas.

Top Solution Authors
Top Kudoed Authors