Forum Discussion
Gen2 Dataflow refreshes breaking since last week
Hello, I and others at my organization are running into issues where most of our Gen2 Dataflows are no longer refreshing. We are getting different erros but one that is common is
Error Code: FailedToInsertATable, Error Details: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: DataSource.Error: Failed to insert a table., InnerException: Unable to create a table on the Lakehouse SQL catalog due to metadata refresh failure, for Lakehouse Id: #{0} and Batch Id: #{1} underlying error #{2}, Underlying error: Unable to create a table on the Lakehouse SQL catalog due to metadata refresh failure, for Lakehouse Id:......Check If there have been recent changes to the schema of your data source, the dataflow may fail during the refresh due to mismatches. Ensure that any schema alterations are reflected in your dataflow's mapping. Issues can arise when referenced queries contain records that still show as [list], even if these queries aren't set to output to a destination. Disabling staging on the main referenced query has resolved similar issues for others. If you're using an on-premises data gateway, ensure it's updated to the latest version. Outdated gateways have been known to cause refresh failures
I think I may have found problem for others that were not aware.
There are hidden artifacts (Lakehouse and such) for dataflows Gen2 when first made. The person who did the first Dataflows in these workspaces has left the company and their account has been deactivated and they were assigned as owner of those artifacts and that seems to be the problem. They are annoying to find but I did eventually find a way to take ownership to fix the issue.
2 Replies
- Vinodh247
Super User
Check If there have been recent changes to the schema of your data source, the dataflow may fail during the refresh due to mismatches. Ensure that any schema alterations are reflected in your dataflow's mapping. Issues can arise when referenced queries contain records that still show as [list], even if these queries aren't set to output to a destination. Disabling staging on the main referenced query has resolved similar issues for others. If you're using an on-premises data gateway, ensure it's updated to the latest version. Outdated gateways have been known to cause refresh failures
I think I may have found problem for others that were not aware.
There are hidden artifacts (Lakehouse and such) for dataflows Gen2 when first made. The person who did the first Dataflows in these workspaces has left the company and their account has been deactivated and they were assigned as owner of those artifacts and that seems to be the problem. They are annoying to find but I did eventually find a way to take ownership to fix the issue.