Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
Joshrodgers123
Advocate V
Advocate V

Re: Datafow Gen2: inconsistent behavior/fails and CU consumption

@miguel - Do you have any guidance/suggestions? We've been struggling to get anything working with DFG2 since Fabric was announced. We're not doing anything crazy - just trying to move relatively small tables from on-prem SQL Server to a warehouse. No transformations at all.

 

We've gone through many different error messages, but the latest one that consistently comes up is below. We've had a support ticket open since November, but they've been bouncing it back and forth and nothing has happened. Our trial is running out and we can't afford to keep trying these on our paid capacity and affect production workloads.

 

104100 Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Web.Contents failed to get contents from 'https://wabi-us-west2-redirect.analysis.windows.net/v1/workspaces' (429): Details: Reason = DataSource.Error;DataSourceKind = Warehouse;DataSourcePath = Warehouse;Url = https://wabi-us-west2-redirect.analysis.windows.net/v1/workspaces;Microsoft.Data.Mashup.Error.Contex... = User GatewayObjectId: 0ff40ade-06f0-4614-be67-a7bc67c40db8

12 REPLIES 12
pqian_MSFT
Microsoft Employee
Microsoft Employee

We are uploading bits for the fix to Download Center, the version is 3000.210.14 - if you go to this version the 429 problem mentioned in this post should be fixed.

Joshrodgers123
Advocate V
Advocate V

I'm happy to report that everything is working well now! It actually resolved itself before updating to the latest gateway. Thanks @pqian_MSFT for the assist!

Anonymous
Not applicable

Hi @Joshrodgers123 ,

Glad to know that your issue got resolved. Please continue using Fabric Community for your further queries.

matkvaid
Helper III
Helper III

I have support ticket created for similar problem. Totaly random CU usage for any of dataflows. They are all ran one after another, initiated in pipeline. 100 000 rows refresh can take 1 minute and consume ~2000CU's, and then another dataflow with 40 000 rows goes for 30mins and consumes 50 000CU's. And that happens totaly random, it can be one or few dataflows, any of them with no matter how many rows. Also this happens with no transformations in dataflow, or for the ones with one filter (wich is folded to SQL). I am waiting for on premise gateway support for pipelines and will try to avoid dataflows as much as possible. 

miguel
Community Admin
Community Admin

Hi!

A couple of suggestions:

  • Please keep the communication with the support team and let us know by the end of the week if there's any updates or if you've reached a resolution
  • Make sure that you're running the latest version of the gateway. At the time of writing this message, the latest version is 3000.210.16
  • There was an issue reported in January that was addressed by the end of that month. It impacted certain cases where the connector was trying to do some internal works getting the list of workspaces and it returned a 429 error.
  • If the problem continues, if you have a way for us to repro the behavior, please share it with us here or through the support ticket

Thanks @miguel. I'll circle back at the end of the week if the issue persists. As of this morning, it still does and it looks like the workspace/429 error you are describing. 

 

As far as repro, all we are doing is loading a few tables from an on-prem SQL server to a warehouse (replace). No additional transformations steps. Largest tables are ~100k rows, but even tables with 20 rows are failing.

 

Gateway is up to date on version 3000.202.16. I know the download page shows 3000.210.16, but when you click download it gives you 3000.202.16.

Are you able to repro this behavior in a different workspace?

Hi @miguel - yes, this is occuring in a few workspaces. The issue still occurs as of 10 minutes ago. Support has not been able to figure anything out yet.

 

This is the error I am seeing most of the time:

 

104100 Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Web.Contents failed to get contents from 'https://wabi-us-west2-redirect.analysis.windows.net/v1/workspaces' (429): Details: Reason = DataSource.Error;DataSourceKind = Warehouse;DataSourcePath = Warehouse;Url = https://wabi-us-west2-redirect.analysis.windows.net/v1/workspaces;Microsoft.Data.Mashup.Error.Contex... = User GatewayObjectId: 

@Joshrodgers123 can you share details from the most recent failure from refresh history page:

 

pqian_MSFT_0-1708021667334.png

 

Joshrodgers123_0-1708021798705.png

Request ID: 5f4a2d42-2483-4b1b-a46b-ec9bd4746bec

Dataflow ID: 9643b30d-5655-450a-a5e3-d942804805a4

Session ID: e79b282a-18a8-f682-a60c-07f88e2dffeb

 

I will say it has gotten better this week. Last week and the previous few months it would fail 90% of the time. It's probably failing 25% of the time this week.

 

 

We investigated this offline with @Joshrodgers123 

There is a fix that will be included in a new gateway version. We'll get it published ASAP.

Hello, please give us update when it this fix will be available, to update gateway. I also have dataflows issues, hoping this will solve mine too

Helpful resources

Announcements
Sept Fabric Carousel

Fabric Monthly Update - September 2024

Check out the September 2024 Fabric update to learn about new features.

Expanding the Data Factory Forums

New forum boards available in Data Factory

Ask questions in Apache Airflow Job and Mirroring.

September Hackathon Carousel

Microsoft Fabric & AI Learning Hackathon

Learn from experts, get hands-on experience, and win awesome prizes.

Sept NL Carousel

Fabric Community Update - September 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors