topic Re: Refresh Token Expiry in Data Engineering

Refresh Token Expiry

smpa01 — Tue, 08 Oct 2024 18:56:43 GMT

I have been runing batch jobs and the token is getting expired ~60/75 min.

// header <CIMultiDictProxy('Content-Length': '104', 'Content-Type': 'application/json; charset=utf-8', 'X-PowerBI-Error-Info': 'TokenExpired', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'X-Frame-Options': 'deny', 'X-Content-Type-Options': 'nosniff', 'Access-Control-Expose-Headers': 'RequestId,X-PowerBI-Error-Info', 'RequestId': 'f098269c-d1b1-473f-86ae-3da25c0b6555', 'Date': 'Tue, 08 Oct 2024 17:15:34 GMT')>

I tried this but this is not consistent. I desparately need the bacth job to run without getting terminated in between and without needing me to babysit.

from datetime import datetime, timezone import pytz import requests def current_hour (): current_time = datetime.now(pytz.timezone('America/Toronto')) hour = current_time.hour text_hour = str(hour) return text_hour token = notebookutils.credentials.getToken('pbi') service_call = requests.get(uri, headers={"authorization": "Bearer " + token,"VariableHeader": current_hour()})

Thank you in advance

Re: Refresh Token Expiry

Anonymous — Wed, 09 Oct 2024 02:29:41 GMT

HI @smpa01,

Perhaps you can try to modify the session timeout of the notebook, you can click on the left-bottom session icon to open session information panel.

Regards,

Xiaoxin Sheng

Re: Refresh Token Expiry

smpa01 — Wed, 09 Oct 2024 03:23:57 GMT

@Anonymous Thanks for this and I will take this out on a spin and let u know.

1 question though - Assuming it works, if I am running batchJob.ipynb with the scheduler at nth hour every day, how can I ensure that it runs with the same custom session timeout (e.g. 90 min) everyday? Asked differently, is the same custom session timeout value applies to the notebook on each run and ca different notebooks have different custom session timeout values ?

Re: Refresh Token Expiry

Anonymous — Thu, 10 Oct 2024 07:20:47 GMT

HI @smpa01,

You can open the workspace setting to modify the spark session timeout.

In addition, you can also consider to use high concurrency mode to running your notebook:
Configure high concurrency mode for notebooks - Microsoft Fabric | Microsoft Learn

Regards,

Xiaoxin Sheng

Re: Refresh Token Expiry

smpa01 — Fri, 11 Oct 2024 16:52:00 GMT

please give me few days to test it out.

Re: Refresh Token Expiry

Anonymous — Mon, 14 Oct 2024 07:06:02 GMT

HI @smpa01,

Sure, you can feel free post here if the above not helps.

Regards,

Xiaoxin Sheng

Re: Refresh Token Expiry

smpa01 — Tue, 15 Oct 2024 01:31:34 GMT

@Anonymous if the notebook is run on a schedule, the token expiry still happens. But if I run the notebook is run from while inside it, then the token does not expire.

I want to run the notebook on a schedule.

Re: Refresh Token Expiry

Anonymous — Tue, 15 Oct 2024 06:34:32 GMT

HI @smpa01,

As you said, it seems like these setting does not enable when you setting schreuder on it. Have you tried to use high concurrency mode to processing your notebook, it should allow to use more resource on it.

Regards,

Xiaoxin Sheng

Re: Refresh Token Expiry

smpa01 — Tue, 15 Oct 2024 17:17:21 GMT

Unfortunately nothing is working, on scheduled job , the token is getting expired around 63-66 min mark (with 90 min timeout session default). I am also not sure if you can defualt a notebook to run with a HC mode. I can only turn on HC mode when inside it.

It is probably because, if I follow this , I get

{ "token_type": "Bearer", "expires_in": 3599, "ext_expires_in": 3599, "access_token": "loremIpsum" }

How can I refresh this token or get access to a reresh token that can be used to API calls nearing 60 min mark

Re: Refresh Token Expiry

FabianSchut — Wed, 16 Oct 2024 05:38:31 GMT

Hi @smpa01,

When do you create the Bearer token and do you re-use the Bearer token over multiple notebook sessions? If you do, you may retrieve the Bearer token in every session instead, so you are less likely to exceed the 1h limit of the Bearer token.