How can I pivot columns but keep duplicates?

So the issue is when pivoting it is assumed there will be unique values after the pivot. If you tell it not to aggregate, and there are not unique items in the unpivoted columns, it won't work.

See this code - put it in Power Query, a blank query.

1) In Power Query, select New Source, then Blank Query
2) On the Home ribbon, select "Advanced Editor" button
3) Remove everything you see, then paste the M code I've given you in that box.
4) Press Done

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMtE3NNM3MjAyUNJRslBw9FVwTszJAbKdMsGUV35mXmoKSM7KwBQorRSrQ1CPT2paCViHIbE6kGwxNCekxzs1J6cSVZMRVosMDfDpsrQyIV4X1EuGBqiazInxk7mVsQFYTywA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [Date = _t, Context = _t, Attendee = _t, Event = _t, Time = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Context", type text}, {"Attendee", type text}, {"Event", type text}, {"Time", type time}}),
    #"Added Joined" = Table.AddColumn(#"Changed Type", "Joined", each if [Event] = "Joined" then [Time] else null, type time),
    #"Filtered for Joined" = Table.SelectRows(#"Added Joined", each ([Joined] <> null)),
    #"Added Left" = Table.AddColumn(#"Changed Type", "Left", each if [Event] = "Left" then [Time] else null, type time),
    #"Filtered for Left" = Table.SelectRows(#"Added Left", each ([Left] <> null)),
    #"Combined Table" = Table.Combine({#"Filtered for Joined", #"Filtered for Left"}),
    #"Removed Other Columns" = Table.SelectColumns(#"Combined Table",{"Date", "Context", "Attendee", "Joined", "Left"})
in
    #"Removed Other Columns"

It will get you to here.

The problem is there is no good way immediately apparent for PQ to know you want row 6 to be combined with row 1, and 7 with 4.

I'd honestly look at this from another angle. Why are you wanting to pivot the Times? Why not leave them in their normalized format and construct your measures to filter based on Join or Leave? In other words, what is your end goal, and let's see if we can get there another way.