Forum Discussion
Pipeline Run RefreshDedupedError, Run button kicking off multiple Activity Runs
Good Day,
I have been attempting to get a datapipeline to run with Dataflow Activity using a parameterized DataflowGen2CICD .
However, when testing I seem to very frequently run into this error rather unhelpful:
Dataflow refresh job failed with status: Failed. Error Info: { errorCode: RefreshDedupedError, message: , requestId: 7d518942-dd2c-4cd3-b9d9-8f760a715e23 }
Suprisingly even though the pipeline is failing, if I go to the "Recent Runs" of the Dataflow item used in the Activity I can see it is succeeding.
The only way to resolve seems to be to wait, and log out of Fabric.
There are a couple of issues here:
1) Time spent "chasing geese" in the figuring out if it is running or not
2) Unable to trust that the UI is reporting what is happening.
Further inspection of the pipeline activity logs makes it look as though the "Run" button in the UI is kicking off multiple runs at the same time (one of which is failing and one succeeds):
Would be great if it could be looked at.
Hi v-lgarikapat
Thanks for the response, for the time being just logging out when this starts and waiting an hour before coming back and trying again. That seems to be the workaround when working in the web UI.
Thanks.
6 Replies
Hi anon97242 ,
Thanks for reaching out to the Microsoft fabric community forum.
You can try the following troubleshooting steps to help resolve the issue.
-
Check Triggering Mechanisms: If the Run button is kicking off multiple runs, ensure there aren't underlying triggers or automated workflows duplicating the execution.
-
Pipeline Dependencies & Parameters: Verify if there are dependencies in the pipeline that could be causing unintended executions.
-
Session Management: Since logging out of Fabric seems to help, it may indicate caching or session-related issues clearing browser cache or switching browsers might be worth testing.
-
Error Logs & Debugging: Reviewing detailed logs in Azure Monitor or other diagnostic tools could provide more insight into what's causing the failure.
I have included the previous solved threads and learning documents; they may help you
Dataflow Gen2 refresh - Microsoft Fabric | Microsoft Learn
Parameters - Microsoft Fabric | Microsoft Learn
Solved: Data pipeline reports failure but Gen2 Dataflow co... - Microsoft Fabric Community
If you're still facing challenges, feel free to let us know we’ll be glad to assist you further.
We look forward to your response.
We appreciate your engagement and thank you for being an active part of the community.
Best regards,
LakshmiNarayana.Hi v-lgarikapat
Thanks for the response, for the time being just logging out when this starts and waiting an hour before coming back and trying again. That seems to be the workaround when working in the web UI.
Thanks.Hi anon97242 ,
Thanks for sharing the workaround you're using for now.Just checking in if the issue is still persisting on your end, please do let us know. We’d be happy to assist further and help explore a more permanent solution if needed.
Thanks again!
Best Regards,
Lakshmi Narayana
Since this seems to be the only resource which google or bing can find on the specific error "RefreshDedupedError", I wonder what is the root cause.
We are experiencing the same error for one of our Dataflows which are run as a steo in a Pipeline.
The Dataflow fails every time the pipeline is started.
So, is the assumption of the topic creator correct, that duplicate execution of the Dataflow causes the Error?
-
