TokenExpired Error Fails Invoke pipeline activity even when underlying pipeline finishes fine

[jankuc]

In Fabric, our orchestration Data pipeline (DP) has an Invoke Pipline Activity that calls another pipeline with some ingestions and other tasks.

So we have orchestration DP that call Ingestion DP. It runs daily and the ingestion DP takes about 90 minutes.

What happens about once a week is that the Ingestion DP finishes successfuly, but the Invoke Activity in the Orchestration DP is marked as Failed with error: TokenExpired with message "Access token has expired, resubmit with a new access token".

What might be the issue and how to work around it? To me it seems that setting multiple retries is not the solution, because then we would run the Ingest DP 2 times.

[echaintreau]

Same problem here, let's hope it will be fix at the end of this month.
The workaround needing to deploy a notebook that check pipeline or notebook status through the API seem not ideal solution.

[Jon_Murphy1]

This exact issue has just occured for me too. Started last night. Same scenario; data pipeline uses the invoke pipeline activty to run another pipeline which takes 90 minutes to run. The invoked pipeline is successful , but the initial pipeline's "Invoke Pipeline" activity fails with the message below. 

{"requestId":"xxxx","errorCode":"TokenExpired","message":"Access token has expired, resubmit with a new access token"}

[jankuc]

I think there might be an error. When DF is polling for the status of the invoked pipeline, they are signing it with a token that expires (lets say after 60minutes) and after that, they dont get the memo that the invoked ppln finished successfuly, but get an error with forbidden access, or some such. 

[Jon_Murphy1]

Yes I fully agree. I've raised a MS Support ticket. If I get a resolution I'll let you know. 

[Jon_Murphy1]

Respnse from Microsoft: "We already have a known issue reported for the similar issue, our Product Team is investigating the issue and I'll let you know when the fix is deployed."

[V-yubandi-msft]

Hi @jankuc ,

Thank you for the update. Once you receive further information from support, kindly share it here. It will be helpful for others in the community who may be facing similar issues.

Regards,

Yugandhar.

[Jon_Murphy1 · Accepted solution]

Response from Microsoft Support: 

PG team has identified the cause, this issue is linked to the use of OAuth connections for pipelines, resulting in expired access tokens that require resubmission with new tokens. The bug work item is already deployed. It should get fixed by the end of this month.

[jankuc]

Do you still see these issues today? For us it seems MS has finally deployed some fix and the Invokes don't fail for us. 

[Jon_Murphy1]

That is good news jankuc. I was contacted by Microsoft yesterday to ask if I was still seeing the issue. I responded with this timeline of failures. So hopfully they applied a fix yesterday or today and I'll stop seeing failures moving forward. 

09/04/2025 – Fail - Access token has expired, resubmit with a new access token

11/04/2025 – Success

12/04/2025 – Success

13/04/2025 – Success

14/04/2025 – Fail - Access token has expired, resubmit with a new access token

15/04/2025 – Fail - Access token has expired, resubmit with a new access token

16/04/2025 – Fail - Access token has expired, resubmit with a new access token

17/04/2025 – Success

18/04/2025 – Success

19/04/2025 – Fail - Access token has expired, resubmit with a new access token

20/04/2025 – Fail - Access token has expired, resubmit with a new access token

21/04/2025 – Fail – Other Unrelated Issue.

22/04/2025 – Success

[Jon_Murphy1]

I continued to have the same issue , but with an internal error message instead of the Access Token Expired Message. It was escalated to the product team and as of last night it appear to now be working correctly. 

Message from Support below:

[kevingauv]

Same problem here.

@Microsoft , one month for the expected resolution... it's quite long!

[jankuc]

Thanks for info. We already have a workaround. A notebook that runs if the InvokeActivity fails which gets the status of the pipeline run through rest api. Looking forward to not needing to use that workaround in the future. 

[jankuc]

Wow, you have much faster MS Support than we do. I've filed it even before posting here.