Forum Discussion
Dataflow Gen2 - failing to load into the lakehouse
Hello. I have a dataflow gen2, fetching data from an azure data factory environment and loading them into a Fabric Lakehouse. It has been working flawlessfly for the last 3 months, but since two weeks, on some of the tables, I get this error:
"There was a problem refreshing the dataflow: "Something went wrong, please try again later. If the error persists, please contact support.". Error code: EntityUserFailure. (Request ID: 6d111afc-11ee-459f-9e31-db4d08c32c44)."
"XXX - ABC Entity Management_WriteToDataDestination: There was a problem refreshing the dataflow: "Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Error in creating lakehouse table in a version, InnerException: Tables can only contain uppercase or lowercase letters, numbers or the underscore character., Underlying error: Tables can only contain uppercase or lowercase letters, numbers or the underscore character. Details: Reason = Expression.Error;ErrorCode = Lakehouse036;Message = Tables can only contain uppercase or lowercase letters, numbers or the underscore character.;Message.Format = Tables can only contain uppercase or lowercase letters, numbers or the underscore character.;ErrorCode = Lakehouse019;InnerErrorCode = Lakehouse021,Lakehouse019;Microsoft.Data.Mashup.Error.Context = User GatewayObjectId: 1ee1c1a1-a11v-32bt-a19b-5ff6d874f432". Error code: 104100. (Request ID: 66d111afc-11ee-459f-9e31-db4d08c32c44)."
The weird part is that there is no consistency between which tables go to error and which ones proceed to being loaded. Sometimes it works sometimes it does not. Also, there are no issue with the naming, as all the columns are respecting the requirements laid out from the error. And above all, it was working flawlessly until few weeks ago and nothing a
6 Replies
Hi gabrifly,
Since the failure occurs during WriteToDataDestination, verify that the destination table (derived from the query name) complies with the supported naming rules and review the destination mapping. Also, review the Lakehouse destination settings to ensure the expected Replace and Dynamic schema behavior is being used and that no recent schema or destination configuration changes have occurred. If the issue continues intermittently across different tables despite valid naming and unchanged configuration, collect the Request ID and refresh history to help correlate the failures and continue the investigation.
Dataflow Gen2 default destination - Microsoft Fabric | Microsoft Learn
Dataflow Gen2 data destinations and managed settings - Microsoft Fabric | Microsoft Learn
Thank you.
Hi gabrifly,
Have you had a chance to review the solution we shared earlier? If the issue persists, feel free to reply so we can help further.
Thank you.
Hello, thanks for taking the time to answer. I already verified that the naming rules are respected. Also, if naming issues were actually behind the failures, I would expect a consistent pattern where the refresh fails everytime. But since this is intermittent, without any edit in the queries in between, I am inclined to rule out the naming. I have the request ID written down but I could not find any error logs besides the one reported in my post.
Hi Mauro89,
Thanks for confirming. Since you've already verified the documented naming requirements and the failures are intermittent across different tables without any query changes, this doesn't appear to match a typical naming validation issue. As an additional isolation step, you could test writing the same Dataflow Gen2 output to a new Lakehouse or a new destination table to determine whether the behavior is specific to the current destination. If the issue persists, I'd recommend opening a Microsoft Support case and providing the captured Request ID, as the available documentation doesn't describe this intermittent WriteToDataDestination/Lakehouse036 behavior and further investigation may require backend diagnostics.
Create a Fabric and Power BI Support Ticket - Power BI | Microsoft Learn
Thank you.
Hello gabrifly,
I have some additonal troubleshooting hint what you can try.
- check the namings of the source (ADF) which is taken bei the DF Gen2. Maybe it refers to some naming changes from there rather than in the destination (assuming by experience that some of the error messages are not quite "helpful").
- check if the DF Gen2 was updated, not only the sources or destinations
- check the data source authentication and if for the failed runs in might correlate with the authentication account (e.g. if you authenticate with a user which is bound to PIM activation this could also lead to issues. again not refering to the actual error message as being helpful)
- check what happends if you define a new destination for the same data like into another Lakehouse, does with work consistently?
Hope this helps troubleshooting.
If so, leave some Kudos or mark the post as solution.
Best regards!
This is generally a lakehouse name and query in dataflowGen2 mismatch, as an example if your query name is
As an example if your destination lakehouse name
Custom_ABC
Query Name
Custom-ABC or Custom.ABCThis looks Microsoft Community case where the reported error was exactly this Lakehouse naming error, and the resolution involved the Dataflow entity/query names, not merely the physical source table.
Further look for column names those generated dynamically from ADF ---->Dataflow---->Lakehouse , although you did check the column names but perfroming further checks could be beneficial.What i mean to say whether the schema entering WriteToDataDestination is deterministic or non deterministic.
At last you can create a test case where you create only table and run the same process from there you debug the actual and isolate the actual problem, and do remember to create new lakehouse where you perform the test, this will narrow down the actual problem, thnaks
If this helps, ✓ Mark as Kudos | Help Other
