RLS measures

Hello, 

I had a question, I have a users as level 1 in the RLS file. However, for email groups, the user is in both a level 1 email group as well as level 2 email group. When I only have the level 1 email group added in security and share it with level 1 email group, he can only see data for the building he is working for. However, whenever, I give access to the level 2 email group, that user starts getting access to everything. Is there a way around this? How would I resolve it?