topic Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption in Dataflow

Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

Joshrodgers123 — Mon, 12 Feb 2024 04:46:55 GMT

miguel - Do you have any guidance/suggestions? We've been struggling to get anything working with DFG2 since Fabric was announced. We're not doing anything crazy - just trying to move relatively small tables from on-prem SQL Server to a warehouse. No transformations at all.

We've gone through many different error messages, but the latest one that consistently comes up is below. We've had a support ticket open since November, but they've been bouncing it back and forth and nothing has happened. Our trial is running out and we can't afford to keep trying these on our paid capacity and affect production workloads.

104100 Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Web.Contents failed to get contents from 'https://wabi-us-west2-redirect.analysis.windows.net/v1/workspaces' (429): Details: Reason = DataSource.Error;DataSourceKind = Warehouse;DataSourcePath = Warehouse;Url = https://wabi-us-west2-redirect.analysis.windows.net/v1/workspaces;Microsoft.Data.Mashup.Error.Context = User GatewayObjectId: 0ff40ade-06f0-4614-be67-a7bc67c40db8

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

miguel — Mon, 12 Feb 2024 06:03:23 GMT

Hi!

A couple of suggestions:

Please keep the communication with the support team and let us know by the end of the week if there's any updates or if you've reached a resolution
Make sure that you're running the latest version of the gateway. At the time of writing this message, the latest version is 3000.210.16
There was an issue reported in January that was addressed by the end of that month. It impacted certain cases where the connector was trying to do some internal works getting the list of workspaces and it returned a 429 error.
If the problem continues, if you have a way for us to repro the behavior, please share it with us here or through the support ticket

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

Joshrodgers123 — Tue, 13 Feb 2024 02:12:44 GMT

Thanks miguel. I'll circle back at the end of the week if the issue persists. As of this morning, it still does and it looks like the workspace/429 error you are describing.

As far as repro, all we are doing is loading a few tables from an on-prem SQL server to a warehouse (replace). No additional transformations steps. Largest tables are ~100k rows, but even tables with 20 rows are failing.

Gateway is up to date on version 3000.202.16. I know the download page shows 3000.210.16, but when you click download it gives you 3000.202.16.

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

matkvaid — Tue, 13 Feb 2024 10:53:04 GMT

I have support ticket created for similar problem. Totaly random CU usage for any of dataflows. They are all ran one after another, initiated in pipeline. 100 000 rows refresh can take 1 minute and consume ~2000CU's, and then another dataflow with 40 000 rows goes for 30mins and consumes 50 000CU's. And that happens totaly random, it can be one or few dataflows, any of them with no matter how many rows. Also this happens with no transformations in dataflow, or for the ones with one filter (wich is folded to SQL). I am waiting for on premise gateway support for pipelines and will try to avoid dataflows as much as possible.

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

miguel — Tue, 13 Feb 2024 14:30:57 GMT

Are you able to repro this behavior in a different workspace?

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

Joshrodgers123 — Thu, 15 Feb 2024 18:04:49 GMT

Hi miguel - yes, this is occuring in a few workspaces. The issue still occurs as of 10 minutes ago. Support has not been able to figure anything out yet.

This is the error I am seeing most of the time:

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

pqian_MSFT — Thu, 15 Feb 2024 18:28:02 GMT

Joshrodgers123 can you share details from the most recent failure from refresh history page:

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

Joshrodgers123 — Thu, 15 Feb 2024 18:32:19 GMT

Request ID: 5f4a2d42-2483-4b1b-a46b-ec9bd4746bec

Dataflow ID: 9643b30d-5655-450a-a5e3-d942804805a4

Session ID: e79b282a-18a8-f682-a60c-07f88e2dffeb

I will say it has gotten better this week. Last week and the previous few months it would fail 90% of the time. It's probably failing 25% of the time this week.

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

pqian_MSFT — Thu, 15 Feb 2024 21:16:25 GMT

We investigated this offline with Joshrodgers123

There is a fix that will be included in a new gateway version. We'll get it published ASAP.

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

matkvaid — Mon, 19 Feb 2024 06:31:49 GMT

Hello, please give us update when it this fix will be available, to update gateway. I also have dataflows issues, hoping this will solve mine too

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

Joshrodgers123 — Thu, 22 Feb 2024 03:12:16 GMT

I'm happy to report that everything is working well now! It actually resolved itself before updating to the latest gateway. Thanks pqian_MSFT for the assist!

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

Anonymous — Thu, 22 Feb 2024 04:21:01 GMT

Hi Joshrodgers123 ,

Glad to know that your issue got resolved. Please continue using Fabric Community for your further queries.

Re: Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

pqian_MSFT — Thu, 22 Feb 2024 23:57:25 GMT

We are uploading bits for the fix to Download Center, the version is 3000.210.14 - if you go to this version the 429 problem mentioned in this post should be fixed.