401 unauthorised when calling API client.Group.GetGroups()

We are embedding a report in a website using the "User Owns Data" method as described here https://github.com/Microsoft/PowerBI-Developer-Samples/tree/master/User%20Owns%20Data.
When we attempt to access the report using a AD guest user, we get a 401 unauthorised exception when calling the client.Groups.GetGroups() API call.

Background:

• This embedded report is successfully working when we log in as the user who published the report and owns the workspace.
• We have added the guest user to the organisations active directory via invite guest
• the guest user is from an external azure AD and is not using a social account.
• This guest user has been allocated a PowerBI Pro license as outlined in https://docs.microsoft.com/en-us/power-bi/service-admin-azure-ad-b2b
• The guest user has had the report shared with them.
• The dataset has been set up with Row Level Security which has been configured for the guest user.
• The guest user can access the report by using the link for that report (from the share screen) i.e. "https://app.powerbi.com/groups/me/reports/[Report Guid]?ctid=[Guid ????]"
• The guest user can not access it from the "WebURL" of the API GetReport response i.e. "https://app.powerbi.com/reports/[report guid]"
• We have attempted to just get the report (in case it was listing groups that was the cause) but client.Reports.GetReportInGroup(groupId, reportId) also causes a 401.
• Authenication for all users is via https://login.windows.net/common/oauth2/authorize/
• We have checked permissions for the registered app and have granted everything but the ones marked "requires admin"

At this point I'm not sure if this is a bug or I've missed something or we are trying to do something that is unsupported, any assistance would be greatly appreciated.

Regards,

Karl