average total matrix

What is the measure being used in that visual?

The reason that's happening is to do with the DAX and how it's evaluated in the total cells, not a setting on the visual

Check out this blog to start with, it might give you an idea of what's going wrong in your DAX:

Obtaining accurate totals in DAX - SQLBI

If this post helps then please consider Accept it as the solution to help the other members find it more quickly.

Matt