RLS USERNAME() in Embedded is being cached, breaking RLS

[dom99]

We have a problem where the USERNAME() is being cached when two users are using power bi embedded at the same time. This has only started happening recently. Its is causing the RLS to break and each user will see the other users data.

Please can it be looked at, it is breaking RLS

[v-yingjl]

Seems like could not reproduce it in my side as far as my test.

Have you tried to use different browsers like Chrome, Edge etc. or use the inprivate mode to check this issue again.

If this issue still happens, you can consider sharing more details about it.

Best Regards,
Community Support Team _ Yingjie Li

[dom99]

Thanks for the reply.

We are able to replicate it now, we are generating a token and passing it to microsoft for embedding the power bi report, but the report is using someone elses token when being viewed.

usually its the token of the person who either last viewed it or viewed it a few minutes ago.

This has only started happening at the end of last week, it is breaking our user based RLS.

Please can it be investigated? Usually takes 2 or 3 refreshes to pick up someone elses credentials

Just to confirm this has been working fine for 3 or 4 years prior to this, however we cant replicate it in the sandbox, only on our system

[indiebitom]

Hi, 

We've ran into the same problem this week, first noticed a bit after 9AM CEST Monday. Our service is not actively used during the weekend, so we believe the problem started happening sometime between Friday night - Monday morning EU time.

In our case, the RLS rules were based on the CUSTOMDATA parameter, but the same thing started happening - concurrent users would sometimes be served data that wasn't their own. 

We've spun a ticket with Microsoft's support on Monday, provided them with a synthetic repro project (built from scratch as a tiny Electron app, embedding a report from a built-from-scratch tiny dataset authored fully by hand). Support has taken the info and liaised with the internal team but so far we haven't heard any timelines or commitments from Microsoft. 


In case anyone else is similarly scrambling to find a fix for this: 
We found that embedding the reports (in javascript) with .embed() instead of .load() and .render() dramatically reduces the repro rate - to the extent that we weren't able to reproduce it at all in our internal testing. We cautiusly shipped a patch to customers earlier today, but we just had a user trip a failsafe after ~4h from release, indicating they were about to be shown data from a different user. So that's a no-go. 

We're rushing to re-architect our solution to shard users into individual datasets now, but overall - all in Microsoft's hands.

[dom99]

We are attempting to quickly re-write it in React, preliminary testing was successful and we couldnt replicate it, but could easily before.

Have you been able to replicate it in the sandbox environment? https://playground.powerbi.com/en-us/

As you say Microsoft are aware and looking into it, please let us know if you have a solution.

[indiebitom]

I'd advise caution with that React rewrite.

Sounds similar to what we experienced - we were using phased rendering .load() and .render() methods (https://docs.microsoft.com/en-us/javascript/api/overview/powerbi/phased-embedding). We noticed we weren't able to reproduce the bug in our app either if we switched to .embed(). In a tiny hand-crafted reproduction project we built, we were also able to consistently leak sessions across users with phased embedding .load() and .render(), but weren't able to reproduce a leak at all if we used .embed(). 

We built a patch with .embed() and shipped it to our customers, but out of abundance of caution, we also introduced a failsafe mechanism that constantly verified if the reports we were about to display had the expected user id (CUSTOMDATA() in our case). That failsafe triggered about 4h after release, aborting the render and pulling our production down again. 

So moral of the story - even if it seems your React rewrite helps, it may just be making the problem less frequent. 

And regarding the Playground repro: 
We also weren't able to reproduce the problem in the Playground. We did manage to reproduce it very consistently in a fully synthetic repro project though (with zero dependency on any of our existing infrastructure and just a handful of visuals, manually typed in tables, and like 40 lines of javascript). 

[dom99]

Thanks for the reply, what mechanism did you use for the failsafe? Sounds like a sensible idea

[indiebitom]

As a failsafe, we added a visual to every report page that would display CUSTOMDATA() (which is what we use for identifying users, you would probably use USERNAME()). As we're embedding the reports in an Electron app, we're able to break into the embedded iframe, grab the actual rendered content of that visual and verify it against the user context that we expect in the app. If it doesn't match, it aborts the load, boots the user, and we get panic alarms to intervene.

We hoped that failsafe would just be a precaution, but alas - it did trigger, so now we're also offline. 

[dom99]

Did you manage to resolve the problem? After the React.js re-write we couldnt replicate and havent had any instances of the fail-safe triggering.

I'm sure you have, but just interested in how it resolved in the end?