Data Flow Gen2 Issue

[Schulle]

Dear colleagues,

I've an issue with a DataFlow Gen2 in Fabric Power BI server environment. The issue occurs only from time to time and I was until not able to find root cause. I would like to ask here before opening a ticket.

Situation: Fabrice Pipeline Starts multiple DataFlow Gen2 in a sequential order, step by step, . One of the Data Flows in the middle produces the error from time to time. 

What the data flow is doing: Pulling data from lakehouse tables, calculates a new table and writes the result back into another table in the same lakehouse.

This is what I get in the pipeline error message:

Error code: 20302

Failure type: User configuration issue

Details: Dataflow refresh transaction failed with status 22

I've googled 20302 and 22 with no useful results.

This is what I get when checking the  error report by looking into the Data Flow Refresh History of the failed Gen2 DF:

CP_TRANSFORM_AMIMMBCD_ACT_RECONCILED_DIV_AGG_WriteToDataDestination: Mashup Exception Data Source Error Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: DataSource.Error: Error in replacing table's content with new data in a version: #{0}., InnerException: #{0}: #{1}, Underlying error: Microsoft SQL: SQL command could not be executed Statement ID: {53BD29EA-2C00-495F-A581-F615229AA19D} | Query hash: 0xE0858D87A194F388 | Distributed request ID: {8F9E1E98-FCAF-4030-8390-71BD9FCF6E9B} Details: Reason = DataSource.Error;Message = Microsoft SQL: SQL command could not be executed Statement ID: {53BD29EA-2C00-495F-A581-F615229AA19D} | Query hash: 0xE0858D87A194F388 | Distributed request ID: {8F9E1E98-FCAF-4030-8390-71BD9FCF6E9B};Detail = [DataSourceKind = "Lakehouse", DataSourcePath = "Lakehouse", DataSourceKind.2 = "SQL", DataSourcePath.2 = "poz3f7falvvung4daaklm6t4pa-kwnkbwwfgyre3hgn4ib3ihiofa.datawarehouse.fabric.microsoft.com;CCO_CapEx_Lakehouse", Message = "SQL command could not be executed#(cr)#(lf)Statement ID: {53BD29EA-2C00-495F-A581-F615229AA19D} | Query hash: 0xE0858D87A194F388 | Distributed request ID: {8F9E1E98-FCAF-4030-8390-71BD9FCF6E9B}", ErrorCode = -2146232060, Number = 100000, Class = 16, State = 1];Message.Format = #{0}: #{1};Message.Parameters = {"Microsoft SQL", "SQL command could not be executed#(cr)#(lf)Statement ID: {53BD29EA-2C00-495F-A581-F615229AA19D} | Query hash: 0xE0858D87A194F388 | Distributed request ID: {8F9E1E98-FCAF-4030-8390-71BD9FCF6E9B}"};ErrorCode = 10478;Microsoft.Data.Mashup.Error.Context = User

I've googled 10478 and Fabric with no useful results.

The Data Flow also works when the overall performance is impacted by heavy load due to other loads, as you can see when checking the different times it took for the runs. Like 35min vs 13 min. Error code in case of failure is always the same as described above.

When I open the data flow to edit directly on Power BI server no error pops up.

[Anonymous]

Hi @Schulle 

Based on the meeage you have offered, the message said that :error in replacing table's content with new data in a version, please check your data source,check if it has changed ,  and you can refer to the following similar thread.

BUG::TRANSIENT::ERROR CODE 20302::user config issu... - Microsoft Fabric Community

Best Regards!

Yolo Zhu

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

[Hakuna_matata]

Hi @Schulle ,
May I suggest that you inspect table and schema, If there have been changes to the table structure or data types in the SQL database, these may affect the query. Ensure that your dataflow logic accounts for any schema changes. 
And if this helps please give it a thumbs up and mark as a solution. Hope it helps!