Ignoring axis context for a measure in chart

I´m trying to use ALLEXCEPT to delete the context for the axis in a chart but not getting there.

Given this data:

I´d like to add a line that shows the average of value across all categories.

I thought that was the use for ALLEXCEPT, the average measure i´m using is:

Average = CALCULATE(SUM(Table1[Value]) / DISTINCTCOUNT(Table1[Category]); ALLEXCEPT(Table1; Table1[Category]))

What am i missing?

EDIT: I should add, there are other fields filtering this visual, so I can´t use ALL, i.e., something like 

Average = SUMX(ALL(Table1);Table1[Value]) / CALCULATE(DISTINCTCOUNT(Table1[Category]); ALL(Table1))

will not work. I want every other filter to be respected except for the X axis.