token used to call REST linked service

[stephanep]

One of our pipelines runs a dataflow that makes, per batch, calls to a REST linked service.
That Rest linked service uses OAuth 2.0 Client Credential authentication.

It seems that when the dataflow runs for too long, the token expires (and the linked service returns 401).

Does it mean that, in a dataflow run, the token endpoint is called before the first call to the REST linked service, and the same token is used for the other calls in that dataflow ?
Is there any way to configure the calls to the linked service in a way that would force the dataflow to get a new token for each call ?

[nilendraFabric · Accepted solution]

Hello @stephanep 

You are correct in Fabric pipelines, when using a REST-linked service with OAuth 2.0 Client Credentials, the token is generated once at the start of the dataflow execution and reused for subsequent calls. This can lead to 401 errors if the token expires mid-execution.

• Access tokens typically expire in 5–10 minutes, while refresh tokens may last up to 30 days.
• Fabric/ADF does not automatically refresh tokens during a dataflow execution. If the pipeline exceeds the token’s lifespan, subsequent calls fail with a 401 error.

Some workarounds 

Use a Web Activity to fetch a new access token before each API call or series of requests

GET https://<token-endpoint>
Headers: Authorization: Basic <base64(client_id:client_secret)>
Body: grant_type=client_credentials

Implement retry logic to detect 401 errors and trigger token refresh

If this is helpful please accept the answer and gives kudos