Inconsistent outcomes between pipeline and dataflow

[ebjim]

I have pipeline that calls several dataflows gen2. Recently, I have seen instances where the dataflow refresh history shows a success but the pipeline show a failure. The pipeline is set to restart once in the event of a failure. What I am seeing is that the pipeline fails both times but the dataflows show one refresh and it is successful. 

Has anyone else experience something like this? Any insights as to the cause/resolution?

Thanks in advance!

[nilendraFabric · Accepted solution]

Hello @ebjim 

Dataflow Gen2 might internally retry operations when encountering transient issues, leading to a successful refresh history despite the pipeline reporting a failure. This could occur due to timeouts or other transient errors during execution

 Worth reading this community discussion 

https://community.fabric.microsoft.com/t5/Data-Pipeline/Data-pipeline-reports-failure-but-Gen2-Dataf...

Hope this helps.

Thanks

[ebjim]

@nilendraFabric  Thanks for referring that post.

[nilendraFabric]

Hello @ebjim happy to help. 

please accept the answer if this resolved the query as it will help community to find the right answer quickly