Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
JoeFields
Frequent Visitor

Custom Connector failing on refreshing data with multiple calls to get a refresh token

We have a custom connector that is using the PKCE OAuth2 flow based around the provided example here and we are having users report occasional issues where they need to sign in again on Power BI Desktop to refresh their data. In looking at API logs on our end, it appears we have some cases where we see repeat calls to use the same refresh token to fetch an updated authentication token in a short period. We speculate there is parallel loading of various tables in Power Query and multiple queries are trying to use the same refresh token to refresh the authentication token at the same time. The first attempt to use the refresh token is successful with a 200 response but all the other calls using the same refresh token receive a 400 status with an error message which breaks the data refresh and the users have to login again for each additional query. This gets messy since often our users might be loading dozens of different tables through our connector.

 

This only appears to be an issue with Power BI Desktop. The refresh is working fine on the Power BI web service.

 

Here is a snippet of the authentication from our connector:

StartLogin = (resourceUrl, state, display) =>
    let
        clientId = getClientIdByRegion(resourceUrl),
        // We'll generate our code verifier using Guids
        codeVerifier = Text.NewGuid() & Text.NewGuid(),
        AuthorizeUrl = authorize_uri & "?" & Uri.BuildQueryString([
            client_id = clientId,
            response_type = "code",
            code_challenge_method = "plain",
            scope="",
            code_challenge = codeVerifier,
            state = state,
            redirect_uri = redirect_uri])
    in
        [
            LoginUri = AuthorizeUrl,
            CallbackUri = redirect_uri,
            WindowHeight = 720,
            WindowWidth = 1024,
            // Need to roundtrip this value to FinishLogin
            Context = codeVerifier
        ];

// The code verifier will be passed in through the context parameter.
FinishLogin = (c, dataSourcePath, context, callbackUri, state) =>
    let
        Parts = Uri.Parts(callbackUri)[Query]
    in
        TokenMethod(dataSourcePath, Parts[code], "authorization_code", context);

TokenMethod = (dataSourcePath, code, grant_type, optional verifier) =>
    let
        // region = Record.Field(dataSourcePath, "region"),
        clientId = getClientIdByRegion(dataSourcePath),
        codeVerifier = if (verifier <> null) then [code_verifier = verifier] else [],
        codeParameter = if (grant_type = "authorization_code") then [ code = code ] else [ refresh_token = code ],
        query = codeVerifier & codeParameter & [
            client_id = clientId,
            grant_type = grant_type,
            redirect_uri = redirect_uri
        ],

        ManualHandlingStatusCodes= {},
        
        Response = Web.Contents(base_path & "/authentication" & "/token", [
            Content = Text.ToBinary(Uri.BuildQueryString(query)),
            Headers = [
                #"Content-type" = "application/x-www-form-urlencoded",
                #"Accept" = "application/json"
            ],
            ManualStatusHandling = ManualHandlingStatusCodes
        ]),
        Parts = Json.Document(Response)
    in
        // check for error in response
        if (Parts[error]? <> null) then 
            error Error.Record(Parts[error], Parts[message]?)
        else
            Parts;
 
Refresh = (ca, resourceUrl, oldCredentials) => TokenMethod(resourceUrl, oldCredentials[refresh_token], "refresh_token");

How can we ensure there is only one call to refresh the authentication token when there are multiple queries that depend on the authentication token?

8 REPLIES 8
balaguru_zoho
New Member

@JoeFields 

did you find any solution for above issue? 

we are getting same issue. 
Oauth Data Refresh called multiple times on token expire scenario · microsoft/vscode-powerquery-sdk ...

AlexZak
Frequent Visitor

Microsoft identity platform and OAuth 2.0 authorization code flow - Microsoft Entra | Microsoft Docs

the refresktoken normally has a live time not a usage count. anyway the refresh call will return a new refresh token if you ask for "offline_access" scope

The custom connector is returning this token to PBI as the body of the result. 

I would recommend you to use fiddler to look for the communication there you will see more information on oauth flow. There you will see the token refresh call and can see if you service provides a new token. And if BPI is using it in the next refresh. 

Yes, here is a Fiddler snapshot of our authentication calls. First one is always 200 but the other calls receive a 400 because the refresh token can only be used once. There is one authentication call for each query from our connector even though the same access token can be used for all queries, Power BI seems to run each query in parallel including the authentication.

 

image.png

can't tell you filtered the between 149 and 152 and 

take a look into the response body of the 200 request it should look like this:

{

"token_type":"Bearer",

"scope":"....",

"expires_in":5332,

"ext_expires_in":5332,

"access_token":"<token>"}

--> There is no request limit only a expirantion time.

and in the 400 request it should give you an oauth error in the return header.

normally if the token is expired your request should look like this:

AlexZak_0-1656578850805.png

 

We have a similar 200 response which contains the access_token, expires_in <seconds>, and refresh_token.

JoeFields_0-1656625436610.png

 

The issue is the new refresh_token is one-time use where as it appears in your case the refresh_token might be re-usable within a narrow window of time.

Here is the 400 response:

JoeFields_1-1656625458740.png

I am looking to see if there is anything in the connector code that we can do to limit the refresh to only one thread/instance or if we can ignore a 400 response as long as we have one 200 response with an updated access_token.

from Microsoft oauth flow - return of a new token refresh:

refresh_tokenA new OAuth 2.0 refresh token. Replace the old refresh token with this newly acquired refresh token to ensure your refresh tokens remain valid for as long as possible.
Note: Only provided if offline_access scope was requested.

 

When you refresh your token a new token is generated that is the default. 

i tried on my side and the BPI is sending and saving the correct token. 

try to implement the advanced OAuth Handling authentication for Power Query connectors - Power Query | Microsoft Docs

 

here is my implementation 

 

redirect_uri = "https://oauth.powerbi.com/views/oauthredirect.html"; // needs to be added as valid redirect URL (Desktop App --> PKCE!) 
logout_uri = "https://login.microsoftonline.com/logout.srf";
baseAuthentificationURL = (tenant) => "https://login.microsoftonline.com/" & tenant & "/oauth2/v2.0";
windowWidth = 1200;
windowHeight = 1000;

//OpenID method to get token endpoints on demand. 
openIDconfig = (dataSourcePath) =>
    let 
        //host = Uri.Parts(dataSourcePath)[Host],
        queryURL = "https://login.microsoftonline.com/" & Settings(dataSourcePath)[tenantName] & "/v2.0/.well-known/openid-configuration",
        queryString =  [],
        response = Web.Contents(queryURL, [
            Headers = [
                #"Accept" = "application/json"
            ],
            ManualStatusHandling = {400} 
        ]),
        body = Json.Document(response),
        result = if (Record.HasFields(body, {"error", "error_description"})) then 
                    error Error.Record(body[error], body[error_description], body)
                 else
                    body
    in
        result;

//will be called as first function on login. 
StartLogin = (clientApplication, dataSourcePath, state, display)  =>
    try
        let
            codeVerifier = Text.NewGuid() & Text.NewGuid(), //needed for PKCE this will remove the need for a client secret.
            authorization_endpoint = openIDconfig(dataSourcePath)[authorization_endpoint],
            AuthorizeUrl =  authorization_endpoint & "?" & Uri.BuildQueryString([
                client_id = Settings(dataSourcePath)[client_id],
                redirect_uri = redirect_uri,
                response_type = "code",
                code_challenge_method = "plain", //needed for PKCE
                code_challenge = codeVerifier, //needed for PKCE
                scope=Settings(dataSourcePath)[client_id] & "/user_impersonation offline_access" //only if offline_access is requested a refresh_token is provided.
                ])

        in
        [
            LoginUri = AuthorizeUrl,
            CallbackUri = redirect_uri,
            WindowHeight = windowHeight,
            WindowWidth = windowWidth,
            Context = codeVerifier
        ]
    otherwise
        let
            message = Text.Format("Start Login Methode Error")
        in
            Diagnostics.Trace(TraceLevel.Error, message, () => error message, true)
;
//after the user has done the MFA this function will be called
FinishLogin = (clientApplication, dataSourcePath, context, callbackUri, state) =>
    let
        parts = Uri.Parts(callbackUri)[Query],
        result = if (Record.HasFields(parts, {"error", "error_description"})) then 
                    error Error.Record(parts[error], parts[error_description], parts)
                 else
                    TokenMethod(dataSourcePath, "authorization_code", parts[code], context)
    in
        result
;

//in the case of first login or token refresh this function will be called and request an (new) access token
TokenMethod = (dataSourcePath, grant_type, tokenFieldValue, optional verifier) =>
    let
        codeVerifier = if (verifier <> null) then [code_verifier = verifier] else [],
        codeParameter = if (grant_type = "authorization_code") then [ code = tokenFieldValue ] else [ refresh_token = tokenFieldValue ],
        queryString = codeVerifier & codeParameter & [
            client_id = Settings(dataSourcePath)[client_id],
            grant_type = grant_type,
            redirect_uri = redirect_uri
        ],
        token_endpoint = openIDconfig(dataSourcePath)[token_endpoint],
        tokenResponse = Web.Contents(token_endpoint, [
            Content = Text.ToBinary(Uri.BuildQueryString(queryString)),
            Headers = [
                #"Content-type" = "application/x-www-form-urlencoded",
                #"Accept" = "application/json"
            ],
            ManualStatusHandling = {400} 
        ]),
        body = Json.Document(tokenResponse),
        result = if (Record.HasFields(body, {"error", "error_description"})) then 
                    error Error.Record(body[error], body[error_description], body)
                 else
                    body
    in
        result;

//Called on token experation
Refresh = (clientApplication, dataSourcePath, oldCredential) => 
    let 
        refreshToken = oldCredential[refresh_token],
        result = TokenMethod(dataSourcePath, "refresh_token", refreshToken)
    in
        result;

//Called on Logout
Logout = (clientApplication, dataSourcePath, accessToken) => logout_uri;

 

AlexZak
Frequent Visitor

Hi,

not sure if this will solve your issue but here is my tokenMethode with works.

TokenMethod = (dataSourcePath, grantType, tokenField, tokenFieldValue, optional verifier) =>
    let
        codeVerifier = if (verifier <> null) then [code_verifier = verifier] else [],
        queryString = codeVerifier & [
            client_id = Settings(dataSourcePath)[client_id],
            grant_type = grantType,
            redirect_uri = redirect_uri,
            code_verifier = verifier

        ],
        queryWithCode = Record.AddField(queryString, tokenField, tokenFieldValue),

        tokenResponse = Web.Contents(baseAuthentificationURL(Settings(dataSourcePath)[tenant]) & "/token", [
            Content = Text.ToBinary(Uri.BuildQueryString(queryWithCode)),
            Headers = [
                #"Content-type" = "application/x-www-form-urlencoded",
                #"Accept" = "application/json"
            ],
            ManualStatusHandling = {400} 
        ]),
        body = Json.Document(tokenResponse),
        result = if (Record.HasFields(body, {"error", "error_description"})) then 
                    error Error.Record(body[error], body[error_description], body)
                 else
                    body
    in
        result;

//Called on token experation
Refresh = (dataSourcePath, refresh_token) => TokenMethod(dataSourcePath, "refresh_token", "refresh_token", refresh_token);

Does the authentication API you are calling allow the refresh token to be used more than once? Our authentication endpoint allows refresh tokens to be used only once. I have seen some other strategies where the authentication endpoint accepts additional calls using the same refresh token if they occur within a short period. Would be best if the connector code could lock the authentication to one thread.

Helpful resources

Announcements
LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.