topic Re: Pipeline Run RefreshDedupedError, Run button kicking off multiple Activity Runs in Data Engineering

Pipeline Run RefreshDedupedError, Run button kicking off multiple Activity Runs

anon97242 — Thu, 12 Jun 2025 16:03:23 GMT

Good Day,
I have been attempting to get a datapipeline to run with Dataflow Activity using a parameterized DataflowGen2CICD .

However, when testing I seem to very frequently run into this error rather unhelpful:

Error code

20302

Failure type User configuration issue

Dataflow refresh job failed with status: Failed. Error Info: { errorCode: RefreshDedupedError, message: , requestId: 7d518942-dd2c-4cd3-b9d9-8f760a715e23 }

Suprisingly even though the pipeline is failing, if I go to the "Recent Runs" of the Dataflow item used in the Activity I can see it is succeeding.

The only way to resolve seems to be to wait, and log out of Fabric.

There are a couple of issues here:
1) Time spent "chasing geese" in the figuring out if it is running or not
2) Unable to trust that the UI is reporting what is happening.

Further inspection of the pipeline activity logs makes it look as though the "Run" button in the UI is kicking off multiple runs at the same time (one of which is failing and one succeeds):

Would be great if it could be looked at.

Re: Pipeline Run RefreshDedupedError, Run button kicking off multiple Activity Runs

v-lgarikapat — Fri, 13 Jun 2025 08:45:33 GMT

Hi @anon97242 ,

Thanks for reaching out to the Microsoft fabric community forum.

You can try the following troubleshooting steps to help resolve the issue.

Check Triggering Mechanisms: If the Run button is kicking off multiple runs, ensure there aren't underlying triggers or automated workflows duplicating the execution.
Pipeline Dependencies & Parameters: Verify if there are dependencies in the pipeline that could be causing unintended executions.
Session Management: Since logging out of Fabric seems to help, it may indicate caching or session-related issues clearing browser cache or switching browsers might be worth testing.
Error Logs & Debugging: Reviewing detailed logs in Azure Monitor or other diagnostic tools could provide more insight into what's causing the failure.

I have included the previous solved threads and learning documents; they may help you

Dataflow Gen2 refresh - Microsoft Fabric | Microsoft Learn

Parameters - Microsoft Fabric | Microsoft Learn

Solved: Data pipeline reports failure but Gen2 Dataflow co... - Microsoft Fabric Community

If you're still facing challenges, feel free to let us know we’ll be glad to assist you further.

We look forward to your response.

We appreciate your engagement and thank you for being an active part of the community.

Best regards,
LakshmiNarayana.

Re: Pipeline Run RefreshDedupedError, Run button kicking off multiple Activity Runs

anon97242 — Fri, 13 Jun 2025 15:09:39 GMT

Hi @v-lgarikapat
Thanks for the response, for the time being just logging out when this starts and waiting an hour before coming back and trying again. That seems to be the workaround when working in the web UI.
Thanks.

Re: Pipeline Run RefreshDedupedError, Run button kicking off multiple Activity Runs

v-lgarikapat — Mon, 16 Jun 2025 10:05:01 GMT

Hi @anon97242 ,

Thanks for sharing the workaround you're using for now.

Just checking in if the issue is still persisting on your end, please do let us know. We’d be happy to assist further and help explore a more permanent solution if needed.

Thanks again!

Best Regards,

Lakshmi Narayana

Re: Pipeline Run RefreshDedupedError, Run button kicking off multiple Activity Runs

v-lgarikapat — Thu, 19 Jun 2025 08:15:39 GMT

Hi @anon97242 ,

If your issue has been resolved, please consider marking the most helpful reply as the accepted solution. This helps other community members who may encounter the same issue to find answers more efficiently.

If you're still facing challenges, feel free to let us know we’ll be glad to assist you further.

Looking forward to your response.

Best regards,
LakshmiNarayana.

Re: Pipeline Run RefreshDedupedError, Run button kicking off multiple Activity Runs

v-lgarikapat — Mon, 23 Jun 2025 09:49:16 GMT

Hi @anon97242 ,

If you're still facing challenges, feel free to let us know we’ll be glad to assist you further.

Looking forward to your response.

Best regards,
LakshmiNarayana.

Re: Pipeline Run RefreshDedupedError, Run button kicking off multiple Activity Runs

Maik_Seyring — Thu, 07 Aug 2025 10:58:06 GMT

Since this seems to be the only resource which google or bing can find on the specific error "RefreshDedupedError", I wonder what is the root cause.

We are experiencing the same error for one of our Dataflows which are run as a steo in a Pipeline.

The Dataflow fails every time the pipeline is started.

So, is the assumption of the topic creator correct, that duplicate execution of the Dataflow causes the Error?