RLS - All users or nothing?

Hello Data Enthusiasts,

Question about PBI RLS. 

I created a new role (limited view of dataset) thinking that I can just add 50 users into it, put them into their own audience and that would be OK. Then I test, those not in the role now have 'access denied' and cannot see what they once saw before RLS, which was everything across ALL reports using the same data model.

 So, It appears that once you activate RLS on the data model you have to place ALL USERS into a role, so I would have to create an 'ALL Access' group as well. This obviously is slight inconvenience.

Is this really what needs to be done? or am I missing something?

I assumed those not in RLS would default to being able to see ALL.

Would a seperate data model be better?

Thanks!