Navigating through relationships

Hi community!

I have a fairly basic question I can't wrap my head around.

Here's my object model:

If I try to put in a table User ID, Permission Set Assignment ID, Permission Set ID and Field Permissions ID, I get a relationship error.

I would have assumed that I would have had a table with User IDs, their related Permission Set Assignments (therefore, User lines would be demultiplied), each Permission Set Assignments' related Permission Set, and for each Permission Set, all of its related Field Permissions. Something like:

If I have the first three columns (to the left) it works, but as soon as I add the last column, which should demultiply lines, it gives me a relationship error.

Any clue what I am not understanding here?

This conceptual question is bugging my mind as it seems trivial but I can't seem to figure out the logic of it.

Thanks a lot!