How to "double" filter?

Yes, this stuff is complex when you first look at a formula.  I cover this in Chapter 14 of my book, but here is the inside running.

First some Definitions

Aggregate function means any of the aggregate functions including SUM, MAX, MIN, AVERAGE, COUNT.  They operate over columns of data providing typically a single scalar value result. https://en.wikipedia.org/wiki/Aggregate_function

When I say "Naked Column", I mean any reference to column in the form TableName[Column Name]  without any aggregate function wrapped around it.

So with these definitions in mind, let's move on.

Explanation

If I write a FILTER formula in pseudo code, it would read like this.

=Filter(my table, where my table[column] has a filter applied that is equal to the Maximum value in the current filter context )

Inside a filter (or any other iterator for that matter), you need to interpret the formula as follows:

1. When you see a “naked column”, it is referring to the column in the table.
2. When you see an aggregate function, it is referring to the current filter context. So in effect the Aggregate functions can “read” the current filter context from a visualisation.  

Reading the Current Filter Context

You need to learn to “read filter context” from your visualisations as one of the first and most important skills in learning to write DAX.  Take the following image and take note of the highlighted cell. 

You need to understand which filters (if any) are impacting this cell BEFORE the calculation of the cell is executed.  So what is the filter context of this cell?  There are 2 filters applied in this case, one is calendar year = 2002 (from Columns in the pivot) and another on Calendar Month = “September” (from Rows in the pivot).  They work together to create a filter context of “September 2002”.  So given the initial filter context is September 2002, what is MAX(Calendar[Date]) in this filter context?  Well because an Aggregate function will respect the filter context, you can infer that it will return the maximum of Date for the current filter context “September 2002”, which of course is 30th Sept 2002.

So new question - what is MAX(Calendar[Month Number]) in the current filter context?  Well given the filter context is September 2002, there is only a single Month Number in the current filter context, and hence MAX(Calendar[Month Number]) = 9.  Note that MIN(Calendar[Month Number]) also = 9, as does AVERAGE and SUM. 

New Question.  what is MAX(Calendar[Year]) in the current filter context?  Once again, there is only 1 year, so MAX, MIN, SUM, AVERAGE all will result in 2002.

This is what I mean by Aggregate functions can “read” the current filter context.  In the case where there is only 1 value in the current filter context, you can use any Aggregate function to “read” the value of the current filter context.

Now, imagine you are applying this new filter you have "read" to a table

Imagine I asked you to apply a filter to the calendar table so that Calendar Year = 2002 and a second filter Calendar Month  = September.  Well you can actually simulate this by going into the back end in Power Pivot, go to the calendar table, find the “year” column and place a filter on “2002”.  Then go to the Month Name column and filter it for “September”.  What will you have left?   You would have 30 days from 1 Sept 2002 through 30 Sept 2002 visible in the table.  So what is MAX(Calendar[Date]) of this filter context?  Clearly it is 30 Sept 2002.  You should learn to "Imagine" this filtering in the back end when you read the filter context from your visualisation. I am using Excel, but it is the same with Power BI.

So back to your formula

Measure = CALCULATE(COUNT('To'[Index]), FILTER('To','To'[Index]= MAX('To'[Index])))             

This formula says

First go and filter the "TO" table.  When you get there, I want you to apply a new filter to the TO[Index] Column to be equal to the Maximum Value you can “See” in the current filter context.  The naked column on the left is referring to which column to apply the filter to, and the MAX() is “reading” the value from the current filter context that needs to be applied.  In the case where there is only a SINGLE value in the current filter context, you could use MIN, MAX, AVERAGE, SUM to “read” the value of the INDEX column.

Note

The following formula is equivalent.  

Measure = CALCULATE(COUNT('To'[Index]), FILTER('To',MAX('To'[Index]) = To'[Index]))   

The point is, it doesn't matter which is on the left or right.  What matters is the naked columns and the aggregate functions.

Now I haven't read all the previous posts, but based on the OP, I would suggest the following approach.

1.  Create a table of all the locations

2.  Don't join this table to your data table.

3.  Write a formula that applies 2 filters in a logical OR.  Something like this

=if(hasonevalue(CitiesTable[City]),
     calculate(Sum(DataTable[VisitorCount],
          filter(DataTable,
                  DataTable[From_City]=Values(CitiesTable[City] || 
                       DateTable[To_City]=Values(CitiesTable[City]
                 )
     )
)

The first filter uses VALUES to harvest the currently selected City from the current filter context.  It then applies a filter on the FROM column.  The second does the same on the TO column.  After both filter are applied, the remaining rows in the data table are used in the formula.  The HASONEVALUE first checks to make sure you have a single filter applied to the CitiesTable.  There is no point trying to "harvest" VALUES(CitiesTable[City]) if  you have more than 1 city selected.