Dataflow Gen2 :Unable to create a table on the Lakehouse SQL catalog due to metadata refresh failure

[mandania]

My DF Gen2 pipeline had been running smoothly for few months now and suddently after 15th november it stopped working (MS Fabric had released an update??).  I have tried everything to fix the problem but its in vain. This is an internal error that only microsoft can help.  this is the error message if it help. 

"kontitrak: Error Code: TridentLakeClientUserException, Error Details: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: DataSource.Error: Failed to insert a table., InnerException: Unable to create a table on the Lakehouse SQL catalog due to metadata refresh failure, for Lakehouse Id: #{0} and Batch Id: #{1}. underlying error code: '#{2}', error: #{3}, Underlying error: Unable to create a table on the Lakehouse SQL catalog due to metadata refresh failure, for Lakehouse Id: b7314f1d-abb8-40b7-b722-6f6d45wc7144 and Batch Id: b7314f1d-abb8-40b7-b722-45w23qwc714@22c30cb9-4b0b-4a66-8175-14c40f76a01a$2024-12-03T11:40:39.4535432Z@f296f27b-ce8a-461a-b97e-45w23qwc714. underlying error code: 'TridentLakeClientUserException', error: [error=[code=TridentLakeClientUserException,pbi.error=[code=TridentLakeClientUserException,parameters=[ErrorMessage=...,HttpStatusCode=...],details={},exceptionCulprit=1]]] Details: Reason = DataSource.Error;ErrorCode = Lakehouse036;Message = Unable to create a table on the Lakehouse SQL catalog due to metadata refresh failure, for Lakehouse Id: b7314f1d-abb8-40b7-b722-45w23qwc714 and Batch Id: b7314f1d-abb8-40b7-b722-45w23qwc714@22c30cb9-4b0b-4a66-8175-14c40f76a01a$2024-12-03T11:40:39.4535432Z@f296f27b-ce8a-461a-b97e-45w23qwc714. underlying error code: 'TridentLakeClientUserException', error: [error=[code=TridentLakeClientUserException,pbi.error=[code=TridentLakeClientUserException,parameters=[ErrorMessage=...,HttpStatusCode=...],details={},exceptionCulprit=1]]];Detail = [error = [...]];Message.Format = Unable to create a table on the Lakehouse SQL catalog due to metadata refresh failure, for Lakehouse Id: #{0} and Batch Id: #{1}. underlying error code: '#{2}', error: #{3};Message.Parameters = {"b7314f1d-abb8-40b7-b722-45w23qwc714", "b7314f1d-abb8-40b7-bgh6-45w23qwc714@22c30cb9-4b0b-4a66-8175-14c40f76a01a$2024-12-03T11:40:39.4535432Z@f296f27b-ce8a-461a-b97e-45w23qwc714", "TridentLakeClientUserException", "[error=[code=TridentLakeClientUserException,pbi.error=[code=TridentLakeClientUserException,parameters=[ErrorMessage=...,HttpStatusCode=...],details={},exceptionCulprit=1]]]"};ErrorCode = Lakehouse045;Microsoft.Data.Mashup.Error.Context = System (Request ID: 7f39f621-9d5a-4d00-93c9-d2cc81aa8e14)."

[lbendlin · Accepted solution]

If you have a Pro license you can open a Pro ticket at https://admin.powerplatform.microsoft.com/newsupportticket/powerbi
Otherwise you can raise an issue at https://community.fabric.microsoft.com/t5/Issues/idb-p/Issues .

[mandania]

I was able to get some help from the fabric support and they managed to solve the problem. This is what they replied "In the previous cases, Product team has identified this bug is caused due to ownership of AD token failure and they have resolved it and took actions so that it will not occur again" My guess is this, the lakehouse that was throwing the error was owned by one of my other colleague who no longer works for us. We had removed his account (that was used to create the LH) and i think this caused a chain of error. unfortunately LH ownership is not something that you can change on fly, so you have to keep in mind that if the account holder is no longer with the compnay , you should probably not remove this person account until fabric comes with a solution.

[v-kongfanf-msft]

Hi @mandania ,

Your comments and feedback are valuable to us and very much appreciated.

Best regards,

Adamk Kong

[v-kongfanf-msft · Accepted solution]

Hi @mandania ,

Based on the error message and the information provided, the issue appears to be related to a failed metadata refresh when attempting to create a table in the Lakehouse SQL Catalog.

The error message instructs TridentLakeClientUserException that if you already have the table in Lakehouse, then simply drag and drop it and recreate it. You must have the exact same schema, or if the schema of the data source has changed, then you may need to update the mapping of the dataflow to reflect the new structure.

Best Regards,
Adamk Kong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[mandania]

Unfortunately i had to create a new workspace and recreate everything from the beginning. Dropping and recreating the tables was not an option since i was not able to do it. I sincerely hope that Fabric team fixes this issue because its not a very good experience for us.

[v-kongfanf-msft]

Hi @mandania ,

Thank you very much for your feedback, we will take this under advisement and get this back to the product team!

Best regards,

Adamk Kong

[lbendlin · Accepted solution]

If you have a Pro license you can open a Pro ticket at https://admin.powerplatform.microsoft.com/newsupportticket/powerbi
Otherwise you can raise an issue at https://community.fabric.microsoft.com/t5/Issues/idb-p/Issues .

[mandania]

Thank you @lbendlin