Unable to Filter Ranks correctly when used with additional two dimensions in the Matrix Visual

Hi Gurus,

I have a matrix visual which looks some thing like this >

I am doing ranking on Cities with the Category on Columns and Segment in Rows. The Dax i have used for generating the Rank function is below 

City Rankx =

CALCULATE(

RANKX(ALLselected('Sample - Superstore'[City]),[Sales Measure]),

ALLSELECTED('Dim Category'[Category]),ALLSELECTED('Dim Segment'[Segment]))

Everything seems fine , except when iam filtering the City Rankx using either a controller (basically using an if condition) or directly from visual it is behaving odd and showing not the expected results.

For example , When i filter City Ranks  is less than or equal = 4, I am getting output as 2,2,3,4 unstead of 1,2,3,4

Pbix Link >https://drive.google.com/file/d/1gRRX6x_XoJsDLyW_AjqZMWAZ56bVSMsG/view?usp=share_link