Anonymous as per the documentation, there are two ways, either the user be in the app workspace or a dashboard with a tile from the underlying report be shared with them.
I'm having a similar issue, if I add the user (external guest) to the app workspace then the RLS is overridden.
When I share the dashboard, the end user is able to view the dashboard and underlying report, but not when the report is embedded in Sharepoint Online using the webpart.
Rather than just the blinking logo, I get a "This content isn't available" error, and a semi-colon above the report. On inspecting in Chrome and Mozilla, it seems like it is an authentication error, maybe the token is not being passed from SPO to PBI?
We are using Power BI Embedded.
I have tried multiple browsers, with the same result.
EDIT: Looks like someone else also mentioned a simialar issue: https://github.com/MicrosoftDocs/powerbi-docs/issues/248