Latency in Workspace Permission Changes using Test As Role

[swise001]

I've got a test PBI Report with Row Level Security enabled - in a workspace.  

Yesterday I 'accidently' added a colleague to the workspace as a "Member" - and seconds later - realizing my mistake - changed his permission to "Viewer".  

When I access the "Test As Role" feature in Power BI Service for this report dataset - and I test this particular user - it is showing me that this employee can still see "everything" - as if he was not impacted by Row Level Security (a virtue of once being a member in the workspace).  

I've found that there can sometimes be latency between the time that I switch the permissions in the Workspace Access  (from Member to Viewer) - to the time those changes are reflected in the "Test as Role".  However, recently - it is now taking over 24 hours and the changes from Member to Viewer are still not reflected in the "Test as Role" when I explicitly type this employees email.  (I know this because I have exposed the USERPRINCIPALNAME() formula - which would pick up his email he is exposed to RLS rules.  

Anyone else experiencing this or can duplicate it?

[v-lili6-msft]

hi  @swise001 

You may refer to this similar post:

https://community.powerbi.com/t5/Service/quot-Test-as-Role-quot-function-is-not-consistently-overrid...

Hope it could give you some help.

Regards,

Lin

[lbendlin]

What happens when you remove the user from the workspace and add them back as Viewer?

[swise001]

@lbendlin 

I've removed the user from the workspace entirely - and the "Test as Role" feature still responds as if this user has "Member" access to the workspace.  (At this point - you would expect the test to fail - since the user no longer has access to the workspace OR dataset).  

If I publish the dataset/report to a new workspace - it operates as expected with this user.   However, if I start to change the permissions of the user in the workspace - again there seems to be a lag on how long those permissions take to reflect in the "Test as Role" functionality.  

Changing the user from Viewer to Member - 'should' remove RLS when using 'test-as-role' for that user.  But it doesn't (at least not in a few minutes).  

It's more a general comment about what 'expectations' we should have when changing permissions in a workspace - and those changes being reflected in the "Test as Role" capability.  

[MattB-Motive]

Experienced the same thing today.

I know this is an old topic, but wanted to post this link here for anyone in the future looking for more info about when the roles actually update: Re: RLS doesn't update with changes in workspace a... - Microsoft Fabric Community

[lbendlin]

I would upgrade that "comment"  to a "concern".  If you have a Pro license you probably will want to raise a ticket with Microsoft and have them clarify the actual policies.