Pipeline Run RefreshDedupedError, Run button kicking off multiple Activity Runs

[anon97242]

Good Day, 
I have been attempting to get a datapipeline to run with  Dataflow Activity  using a parameterized  DataflowGen2CICD .

However, when testing I seem to very frequently run into this error rather unhelpful:

Error code

20302

Failure type User configuration issue

Dataflow refresh job failed with status: Failed. Error Info: { errorCode: RefreshDedupedError, message: , requestId: 7d518942-dd2c-4cd3-b9d9-8f760a715e23 }

Suprisingly even though the pipeline is failing, if I go to the "Recent Runs" of the Dataflow item used in the Activity I can see it is succeeding. 

The only way to resolve seems to be to wait, and log out of Fabric. 

There are a couple of issues here:
1) Time spent "chasing geese" in the figuring out if it is running or not
2) Unable to trust that the UI is reporting what is happening.

Further inspection of the pipeline activity logs makes it look as though the "Run" button in the UI is kicking off multiple runs at the same time (one of which is failing and one succeeds):

Would be great if it could be looked at.

[v-lgarikapat]

Hi @anon97242 ,

Thanks for reaching out to the Microsoft fabric community forum.

You can try the following troubleshooting steps to help resolve the issue.

• Check Triggering Mechanisms: If the Run button is kicking off multiple runs, ensure there aren't underlying triggers or automated workflows duplicating the execution.

• Pipeline Dependencies & Parameters: Verify if there are dependencies in the pipeline that could be causing unintended executions.

• Session Management: Since logging out of Fabric seems to help, it may indicate caching or session-related issues clearing browser cache or switching browsers might be worth testing.

• Error Logs & Debugging: Reviewing detailed logs in Azure Monitor or other diagnostic tools could provide more insight into what's causing the failure.

I have included the previous solved threads and learning documents; they may help you

Dataflow Gen2 refresh - Microsoft Fabric | Microsoft Learn

Parameters - Microsoft Fabric | Microsoft Learn

Solved: Data pipeline reports failure but Gen2 Dataflow co... - Microsoft Fabric Community

If you're still facing challenges, feel free to let us know we’ll be glad to assist you further.

We look forward to your response.

We appreciate your engagement and thank you for being an active part of the community.

Best regards,
LakshmiNarayana.

[Maik_Seyring]

Since this seems to be the only resource which google or bing can find on the specific error "RefreshDedupedError", I wonder what is the root cause.

We are experiencing the same error for one of our Dataflows which are run as a steo in a Pipeline.

The Dataflow fails every time the pipeline is started.

So, is the assumption of the topic creator correct, that duplicate execution of the Dataflow causes the Error?

[anon97242 · Accepted solution]

Hi @v-lgarikapat 
Thanks for the response, for the time being just logging out when this starts and waiting an hour before coming back and trying again.  That seems to be the workaround when working in the web UI.
Thanks.

[v-lgarikapat]

Hi @anon97242 ,

Thanks for sharing the workaround you're using for now.

Just checking in if the issue is still persisting on your end, please do let us know. We’d be happy to assist further and help explore a more permanent solution if needed.

Thanks again!

Best Regards,

Lakshmi Narayana

[v-lgarikapat]

Hi @anon97242 ,

If your issue has been resolved, please consider marking the most helpful reply as the accepted solution. This helps other community members who may encounter the same issue to find answers more efficiently.

If you're still facing challenges, feel free to let us know we’ll be glad to assist you further.

Looking forward to your response.

Best regards,
LakshmiNarayana.

[v-lgarikapat]

Hi @anon97242 ,

If your issue has been resolved, please consider marking the most helpful reply as the accepted solution. This helps other community members who may encounter the same issue to find answers more efficiently.

If you're still facing challenges, feel free to let us know we’ll be glad to assist you further.

Looking forward to your response.

Best regards,
LakshmiNarayana.