Previous date according filter

Hi,

I have a list of project milestones. Format is as follows:

ProjectID    Milestone             Milestone Date

I am wishing to create a measure which calculates the previous milestone date for a project. The desired result would be as follows.

However I would like this measure to calcuate the previous milestone based on what is filtered. So for example if only A, C and E are filtered the result would be as follows:

The below creates what I am looking for in the first example but I am stumpted as to what to do for the second.

PreviousMilestoneDate = 
VAR MilestoneDate = MAX(Fact_Milestones[MilestoneDate])
VAR ProjectID = MAX(Fact_Milestones[ProjectID])


RETURN
MAXX(
	FILTER(
		ALL(Fact_Milestones),
		Fact_Milestones[MilestoneDate] < MilestoneDate &&
		Fact_Milestones[ProjectID] = ProjectID
	),
	Fact_Milestones[MilestoneDate]
) 

Many Thanks