Custom data connector using oauth2 not refreshing access token

[redec]

 I've got a custom data connector which authenticates to the backing service using oauth2. It authenticates fine, and runs fine for smaller extracts....but the backing service's access tokens are only valid for an hour, so any extract which takes longer than an hour fails with credential errors after an hour. I assumed PowerBI would automatically refresh the access token as needed, is this not the case? I'm providing it the refresh method, and it knows when the access token is going to expire...is there something special I need to do to get it to actually perform the refresh before the access token expires?

[dbeavon3]

@redec 

Its been two weeks of working on a support ticket about this. 

I have now learned that it is a known issue that oauth access tokens will not be refreshed in the enterprise gateway. Oddly the tokens are refreshed successfully within the PBI desktop environment. Just not in the gateway.

If a developer creates a custom connector that works fine in the desktop, they should be able to assume it will work fine in the enterprise gateway as well. It is very frustrating to spend so many weeks digging into this, only to find out that it is a known issue for which a fix hasn't been prioritized yet:

https://docs.microsoft.com/en-us/power-bi/connect-data/refresh-troubleshooting-refresh-scenarios#ref...

"Microsoft is investigating a solution that allows the data loading process to refresh the token and continue.."

Given the fact that this "AAD" auth-kind is using Microsoft's own identity services, you would expect a Microsoft product would not have a compatibility/interop problem. I am probably not the only Microsoft customer who would assume the problem is in my solution, thereby wasting a ton of time. Given these recent experiences with the Power BI product and support, I can't help but think that even a third party tool (Tableau) would have better compatibility with AAD than Power BI does.

[dbeavon3]

Microsoft updated a page to say that oauth2 tokens expire after an hour in the gateway.

https://docs.microsoft.com/en-us/data-integration/gateway/service-gateway-onprem-faq#why-do-i-get-th...

This is buggy behavior. And it is very different than what we experience in the PBI desktop. Ideally the enterprise gateway would work better than the PBI desktop, rather than worse.

Based on the inspection of the underlying code, the AAD tokens are supposed to be refreshed. I'm not sure why there is so much friction going on between a popular Microsoft product, and the Microsoft AAD identities.

[dbeavon3]

Two years later, and there is very little progress.  I did notice a comment that says dataflows may not suffer the same problems as datasets:

"
Dataflows specifically were recently updated to support token refresh, so they are no longer subject to the 1 hour timeout. However, other artifacts, such as pipelines, copy jobs, or semantic models, are still subject to the limitation."


... Unfortunately the reason dataflows aren't a good option, is because we can't use them with custom connectors.  There is no support for custom connectors in dataflows.  That incompatibility has been in place for many years, and Microsoft keeps pushing out their promised ETA.  I believe the current ETA is end of 2025.

[redec]

It looks like the problem only occurs when triggered from powerbi server.  When triggered from powerbi desktop it works as expected - it gets a new access token before the old one expires.  When triggered from powerbi server it just gets an access token at the beginning and then never again.

I feel like I must be missing something obvious, because it seems like this would be a big problem for a lot of people...