Averaging Minimums not Working Correctly

So ultimately what I want to do is calculate the average minimum time between two different events at each location.

In order to do this, I started with making a column:

NOR2CargoLoading = datediff('Time Log'[Notice of Readiness Tendered],'Time Log'[Commenced Cargo Loading],minute)

After this, I needed to find the minimum time per voyage.

Min of NOR2CargoLoading min per Voyage # =
MINX(
 KEEPFILTERS(VALUES('Time Log'[Voyage #])),
 CALCULATE(MIN('Time Log'[NOR2CargoLoading]))
)

At this point, everything is working correctly. Next I tried to average the minimums and this is where things start to go south:

Min of NOR2CargoLoading min per Voyage # average per Location =
AVERAGEX(
 KEEPFILTERS(VALUES('Time Log'[Location])),
 CALCULATE('Time Log'[Min of NOR2CargoLoading min per Voyage #])
)

Instead of calculating the average for each location, it is calculating the minimum for each location. Any advice?