Merge query slows down when I expand user fields using SharePoint lists

Greetings,

I have a merge query that does a join to a list with 5 user fields. When I expand those user fields to find the first name and last name the query slows way down. Is this because the user fields are basically lookups? Is this normal?

I tried to optimize the query and reduce the number of fields that I return to the bare minimum. It appears when I hit the Step that extracts the names, the query slows down.