Forum Discussion
Data Pipeline : Invoke Pipeline Issue
Hi,
In one of my workspaces I have 2 pipelines, one of which calls the other using the invoke pipeline utility.
The parent pipeline is successfully triggering the child pipeline which is then in turn running the notebooks utilities inside of it.
These notebook utilities all have a retry count of 2, meaning if they fail they'll retry.
My issue is that these notebooks fail on the first run but will succeed on the second run, that first failure is causing the parent pipeline to report a failure.
Even more strangely the child pipeline continues to run until it succeeds, even after a failure has been reported to the parent pipeline.
I have had this set up for months with no problems so I suspect something has changed.
This is the error that is reported from the child pipeline to the parent pipeline.
{"requestId":"XXX-XXX-XXX","errorCode":"TokenExpired","message":"Access token has expired, resubmit with a new access token"}
If someone could offer some advice I'd appreciate it.
Hi Jester_3 ,
The pipeline is failing because the child pipeline is taking longer to complete. To resolve this, I switched to using the Invoke Pipeline (Legacy) activity instead of the Invoke Pipeline (Preview) activity, and the pipeline succeeded.
Please use Invoke Pipeline (Legacy), as the Preview version relies on a linked service, and the access token used by that linked service is expiring during long-running executions.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Thanks & Regards,
Rekha,
CustomerSupportTeam.
12 Replies
Thank you for the response, i'm already using the invoke-pipeline-activity.
Like you suggest, I think i'll contact Mircosoft Support.
Hi Jester_3
Microsoft has released a preview version that allows invoking pipelines across Fabric workspaces and monitoring child pipelines. Try using this once
https://learn.microsoft.com/en-us/fabric/data-factory/invoke-pipeline-activityIf the issue persists, it may be worth contacting Microsoft Support, as there could be a platform-specific issue or limitation that needs to be addressed
Hi Jester_3 ,
Thank you for reaching out to Microsoft Fabric Community.
In the "Invoke Pipeline" activity, please ensure that the "Wait on completion" option is checked so that the parent pipeline only succeeds after the child pipeline completes successfully. PFBS for reference.
I tested this by creating a script that fails on the first run and succeeds on the second. Then, I called this pipeline from another pipeline using the "Invoke Pipeline" activity with "Wait on completion" enabled, and the parent pipeline completed successfully.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Thanks & Regards,
Rekha,
CustomerSupportTeam.
Hi v-sathmakuri
I checked that the wait for completion setting was enabled on the invoke-pipeline-activity and it was.
Thanks for the suggestion.
Hi Jester_3 ,
Could you please check whether the pipeline you're trying to run has succeeded? Also, please review the latest run of this pipeline using the Monitor option in Fabric
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Thanks & Regards,
Rekha,
CustomerSupportTeam.
