Forum Discussion

jankuc's avatar
jankuc
Advocate I
1 year ago
Solved

TokenExpired Error Fails Invoke pipeline activity even when underlying pipeline finishes fine

In Fabric, our orchestration Data pipeline (DP) has an Invoke Pipline Activity that calls another pipeline with some ingestions and other tasks.

So we have orchestration DP that call Ingestion DP. It runs daily and the ingestion DP takes about 90 minutes.

What happens about once a week is that the Ingestion DP finishes successfuly, but the Invoke Activity in the Orchestration DP is marked as Failed with error: TokenExpired with message "Access token has expired, resubmit with a new access token".

What might be the issue and how to work around it? To me it seems that setting multiple retries is not the solution, because then we would run the Ingest DP 2 times.
  • Jon_Murphy1's avatar
    Jon_Murphy1
    1 year ago

    Response from Microsoft Support: 

     

    PG team has identified the cause, this issue is linked to the use of OAuth connections for pipelines, resulting in expired access tokens that require resubmission with new tokens. The bug work item is already deployed. It should get fixed by the end of this month.

14 Replies

  • Jon_Murphy1's avatar
    Jon_Murphy1
    Frequent Visitor

    This exact issue has just occured for me too. Started last night. Same scenario; data pipeline uses the invoke pipeline activty to run another pipeline which takes 90 minutes to run. The invoked pipeline is successful , but the initial pipeline's "Invoke Pipeline" activity fails with the message below. 

     

    {"requestId":"xxxx","errorCode":"TokenExpired","message":"Access token has expired, resubmit with a new access token"}

     

     

    • jankuc's avatar
      jankuc
      Advocate I

      I think there might be an error. When DF is polling for the status of the invoked pipeline, they are signing it with a token that expires (lets say after 60minutes) and after that, they dont get the memo that the invoked ppln finished successfuly, but get an error with forbidden access, or some such. 

      • Jon_Murphy1's avatar
        Jon_Murphy1
        Frequent Visitor

        Yes I fully agree. I've raised a MS Support ticket. If I get a resolution I'll let you know. 

  • Same problem here, let's hope it will be fix at the end of this month.
    The workaround needing to deploy a notebook that check pipeline or notebook status through the API seem not ideal solution.