Forum Discussion

MisterFry's avatar
MisterFry
Icon for Resolver III rankResolver III
5 years ago
Solved

Implementing Binomial Hypothesis Testing significance tests in Power BI (DAX)

This is partly a theory question, and partly an implementation question. My stats is a little rusty...

I am developing a report that is attempting to determine if the difference in occurances between a reference group and a selected group are statistically significant.

So, for example, if something occurs in X of n tests for one group, is it statistically significant than if it 'normally' occurs at a rate of Y of m tests for a different (control) group.

So, my H0 is that the rate is Y of m, per the control group h1 is that it is not the same as the control group. (ideally, I'd like to use a 1-tailed test, depending if the observed occurrence is greater or less than the control, but my current implementation is 2 tailed)

I'd be comfortable with a CI of 80%.

I've got (slightly pseudocode here):

Zscore = 
   VAR pControl = DIVIDE(COUNT([Control occurrences]), COUNT([Control Tests])) RETURN
   VAR pTest = DIVIDE(COUNT([Test occurrences]), COUNT([Test Tests])) RETURN
   VAR controlStandardError = 
      SQRT(
         DIVIDE(
            (pControl * (1-pControl)
            , COUNT([Control Tests])
          )
      ) RETURN
   VAR testStandardError = 
      SQRT(
         DIVIDE(
            (pTest* (1-pTest)
            , COUNT([Test Tests])
          )
      ) RETURN
DIVIDE(
   (pTest - pControl)
   , SQRT(POWER(testStandardError, 2) + POWER(controlStandardError, 2)
)

I'm then calculating:

p-Value = 
   VAR pControl = DIVIDE(COUNT([Control occurrences]), COUNT([Control Tests])) RETURN
   IF(pControl > 0, 
      1 - ABS(NORM.DIST(Zscore, 0, 1, TRUE)
   )

I am then displaying in a table each of my non-null hypotheses and filtering the table such that p-Value is less than 0.1. (2-tailed 80%)

am I on the right track here? Or have I completely bungled the theory on this one?