Forum Discussion
Datafow Gen2: inconsistent behavior/fails and CU consumption
I am having some trouble understanding why a Dataflow Gen2 is failing sometimes.
As you can see from the screenshot of the refresh history the dataflow is working fine for some time (running every day upon scheduled execution) and then for no apparent reason the dataflow fails to refresh.
After manually clicking the refresh button once or twice (without changing the dataflow itself OR changeing the data that is being processed!) the flow eventually refreshes successfully - even if taking much longer (25min compared to usually about 8min) and also consumes much more CU than usually!
It is always a "WriteTODataDestination" activity that fails - the exact error message is:
104100 Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: DataSource.Error: Web.Contents failed to get contents from 'https://wabi-west-europe-d-primary-redirect.analysis.windows.net/v1/workspaces' (429): Details: Reason = DataSource.Error;DataSourceKind = Warehouse;DataSourcePath = Warehouse;Url = https://wabi-west-europe-d-primary-redirect.analysis.windows.net/v1/workspaces;Microsoft.Data.Mashup.Error.Context = User
This dataflow is not using a gateway. The single datasource is a Fabric Lakehouse and the destination is a Fabric Warehouse!
Two things that I find rather troublesome:
1. as the dataflow is unchanged and also the data processed is not changing between a fail and a successful refresh I have a hard time understanding why the dataflow fails and what I can do to prevent it from failing!
2. those fails (that do not seem to be caused by a user error) still do consume quite a number of CU. This is a screenshot from the metrics app filtered to the dataflow in question. You can see that today the consumption is much higher - as it also was 14 days ago. In between those dates the dataflow was working as expected and did consume a significantly lower amount of CU.
I would be interested in understanding why a dataflow sometimes fails - especially if it later runs successfully without changing anything and how this is impacting the CU consumption!
Thank You!
Hi chris__1 , Joshrodgers123
This particular failure (calling /workspaces and receiving 429) is happening due to an internal throttling issue that has been resolved since beginning of this week.
Please monitor these refreshes for a bit and see if you continue to observe failures due to "429" error. If so, let me know and sharing the session\request\dataflow ID from the refresh history screen:
13 Replies
We've been seeing this error on all of our dataflows causing them to fail (and consume our CU). We've had a support ticket open since November for DFG2 issues and no one has been able to fix it.
thank you Joshrodgers123 - its good to know that I am not alone seeing this behavior.
As our trial is running out soon I think its going to be difficult have to convince the right people to put this product into production and to spend money on it. Not being able to understand why a workload fails is one thing - but to even have to pay for it as your CU go up... thats not good.
Hi chris__1
Thanks for using Fabric Community. Apologies for the issue you have been facing.
For handling such scenarios we would need more information regarding your workspace details, session id etc. Our engineering team would better understand the issue and help you. Hence I request you to create a support ticket here:
Microsoft Fabric Support and Status | Microsoft Fabric
Please provide the ticket number here so that I can closely follow up and provide you with a solution.
Thanks.
Hi chris__1 , Joshrodgers123
This particular failure (calling /workspaces and receiving 429) is happening due to an internal throttling issue that has been resolved since beginning of this week.
Please monitor these refreshes for a bit and see if you continue to observe failures due to "429" error. If so, let me know and sharing the session\request\dataflow ID from the refresh history screen:
thanks pqian_MSFT . in fact I did not encounter this error this week. I also could not replicate it when working with support. we did close the ticket and I'll come back if it reoccurs....
