BUG::TRANSIENT::ERROR CODE 20302::user config issue but NOT!

[Element115]

CONTEXT:

A pipeline runs a bunch of DFs.  Testing was going on for more than 24 hours. Run schedule is hourly. One of the first DFs suddenly failed. The DF reads a very, but very small Dim table in replace mode from an on-prem DB.

At the time the PL run history says this DF activity failed, the DF refresh history shows nothing failed:

ISSUE:

Pipeline fails at first dataflow activity with:

Error code: 20302

Failure type: User configuration issue

Details: Dataflow refresh transaction failed with status Unknown

What is strange is that the DF start time from the DF refresh history is off by 55 secs from the activity start time: 

activity run start = 3/27/2024  06:28:01 PM

DF start time = 3/27/2024  06:28:56 PM

Also strange is that after the test run was launched, I ofc did not change any config whatsoever in the middle of testing!  Therefore why is the error reporting a user config issue? Makes no sense whatsoever.  

Finally, this also happened the same day 4 hours before. So in one day, this error occurred at 02:28 PM and then at 06:28 PM. But not since then.

[JoachimSA]

I have the same issue...any progress on this ?

[Element115]

And the new issue I mentioned is resolved.  Wasn't supposed to do a CAST in the T-SQL statement I was feeding the Copy data Activity in the pipeline. If interested, here's the breakdown:  

Solved: Re: HELP::COPY DATA::JPEG BINARY LOAD FAIL - Microsoft Fabric Community

[Element115]

The issue went away on its own for me, but instead got replaced with this new issue:

BUG::COPY DATA::lakehouse displays table while say... - Microsoft Fabric Community

and now as I check the lakehouse, it too magically disappeared on its own.  I am glad the issues are gone, but it's kinda scary because it means anything could happen at any time, magically, breaking all your ETL/ELT workflow.  Not a good predicament to be in, this lack of predictability.

----- EDIT ----

Oops! I spoke ot soon. The issue came back.  Just took a while to update the GUI for the error message to appear.  Bummer.  Still no resolution in sight.

[v-cboorla-msft]

Hi @Element115 

Thanks for using Microsoft Fabric Community.

Apologies for the inconvenience that you are facing here.

The error code 20302 in Microsoft Fabric Dataflows points to a user configuration issue, but the "Unknown" status makes pinpointing the exact cause a little tricky. 

Resource limits within the Data Pipeline might be the cause of this error. If your dataflow is particularly large or complex, exceeding resource limits could lead to similar errors. The official Microsoft Fabric documentation provides details on data pipeline resource limitations for your reference.

You can refer to similar reported issue thread : Data pipeline reports failure but Gen2 Dataflow completes.

If the issue still persists, please do let us know. Glad to help.

I hope this information helps. 

Thanks.

[Element115]

It is obviously not a resource limit issue since the error never occurred again and the pipeline keeps running this DF on an hourly basis with success.  Besides, the DF is only producing a 6 row table with a few columns, so in terms of resources it is utterly insignificant.

[v-cboorla-msft]

Hi @Element115 

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.

Thank you.