Forum Discussion

smpa01's avatar
smpa01
Community Champion
1 year ago

Dataflow gen2 refresh Error

There was a problem refreshing the dataflow: 'Something went wrong, please try again later. If the error persists, please contact support.'. Error code: EntityUserFailure. (Request ID: 460e8fb5-e1f7...
workspaces
v-veshwara-msft's avatar
v-veshwara-msft
Community Support
1 year ago

Hi smpa01 ,
Thanks for engaging with Microsoft Fabric Community.
Just checking in to see if your issue is resolved and if you’ve had a chance to raise a support ticket. If the issue has been resolved, it would be helpful if you could share the resolution steps here for the benefit of the wider community.

 

I also followed the repro steps you shared and was able to observe the same behavior on my end. The error still occurred even after creating a new table and using a new Dataflow Gen2.

Hope the support team is able to help resolve the issue.
Thanks miguel, LuitwielerMSFT and wardy912 for addressing this and sharing valuable insights.

 

Please reach out for further assistance.

Thank you.

  • smpa01's avatar
    smpa01
    Community Champion
    1 year ago

    I’m currently under significant time pressure to deliver a production-ready workflow for my client. Over the past two weeks, I’ve been evaluating Dataflow Gen2 for my use case, but have encountered several critical issues and unexpected behaviors:

     

    - Incremental refresh does not function as expected

    - Refresh operations result in Lakehouse tables with the correct schema and row count, but contain no data

    - No REST API support for triggering refreshes using a service principal (SPN)


    These challenges are difficult to diagnose or address intuitively, making DF Gen2 unreliable for my current complex production scenario.

     

    While I plan to raise a formal support ticket once immediate client deliverables are met, I’ve documented the steps to reproduce these issues (outlined above). My current focus is to design a robust and scalable data engineering solution—ideally leveraging Notebooks as the primary mechanism, rather than relying on low-code artifacts like Dataflows or Pipelines. This is particularly important in the context of enterprise-level BI that integrates data from diverse sources, including on-prem systems, Databricks, Dremio, SharePoint, and others.

     

    In contrast, Dataflow Gen1—though limited to producing flat tables and lacking SQL or Spark transformation capabilities—proved far more stable in practice. From a functional standpoint, it presented minimal issues. The unfortunate limitation is that it cannot insert data into a destination.

     

     

    • v-veshwara-msft's avatar
      v-veshwara-msft
      Community Support
      1 year ago

      Hi smpa01 ,

      Thanks for the detailed follow-up and for sharing your observations. We completely understand the pressure of delivering a stable, production-ready solution, and we appreciate you taking the time to list out the issues and repro steps despite that.

       

      It’s clear that some of the current limitations in Dataflow Gen2, especially around incremental refresh and service principal support for refresh, are making it difficult to use in your production scenario.

       

      When you're able to, raising a support ticket will allow the engineering team to look into the issue more deeply. In the meantime, using notebooks could be a good alternative, especially since they offer more control and flexibility when working with different data sources like Databricks, SharePoint, and on-prem systems.

       

      Thanks again for your feedback - it helps improve the experience for everyone. Feel free to share any further updates here.

    • rubymaya's avatar
      rubymaya
      Helper III
      4 months ago

      Hi,

      Can i know if this is resolved, please? Im having the same problem.

      Trying to get data via OnPrem to destination Lakehouse, but getting the versioning error when i turn on the Incremental Refresh. I had to say that the Inc. Refesh in Gen2 is horrible than the Gen1. (Especially when it needs all step to fold, no native query)

       

      "Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: The value does not support versioning. Details: Reason = Expression.Error;ErrorCode = 10855;Detail = #table({"0soursystem", "MANDT", "lang_key", "plant", "planner_cd", "planner-name"}, {});Microsoft.Data.Mashup.Error.Context = User GatewayObjectId: fb0ad49a-bd40-4089-bb20-21c5da700313". Error code: Mashup Exception Expression Error. (Request ID: 90473d75-c6d4-4065-b82b-4c455c9e6dbc).

  • oe_lopez's avatar
    oe_lopez
    Frequent Visitor
    1 year ago

    I had the same error, but I had a similar dataflow gen 2 working, so I did very basic troubleshooting and removed all non text columns and started adding them and sure enough it was a number column that was giving me the error. Fixed that column and it all worked.

     

    regards,