Forum Discussion

MOVC's avatar
MOVC
Icon for Resolver I rankResolver I
2 years ago

Dataflow Gen2 Failure: Couldn't refresh the entity, transport-level error

This morning one of our Gen2 Dataflows began failing. It takes data from our On Prem SQL Server via the Data Gateway and creates a new table in a Warehouse with the replace method. The error message is as follows:

 

We have had no issues with since it was created about a month ago and there have been no changes to our network, On Prem environment or the source data. Our Gateway is on the previous version which I will update tonight out of work hours to check, but this has not impacted our other Fabric methods and seems unlikely to be the cause seeing as it worked as recently as yesterday.

Any help would be appreciated, but this is not the first time something we've created in Fabric has failed out of the blue and it is becoming difficult to suggest it as a solution with the inconsistency.

 

EDIT: I forgot to mention, I have tried deleting and recreating the Warehouse as other users had suggested but this has not worked.

Dataflows
Need Help

11 Replies

  • v-cboorla-msft's avatar
    v-cboorla-msft
    Icon for Microsoft Employee rankMicrosoft Employee
    2 years ago

    Hi MOVC 

     

    Thanks for using Microsoft Fabric Community.

    Apologies for the inconvenience that you are facing here.

    When using Microsoft Fabric Dataflow Gen2 with an on-premises data gateway, you might encounter issues with the dataflow refresh process. The underlying problem occurs when the gateway is unable to connect to the dataflow staging Lakehouse in order to read the data before copying it to the desired data destination. This issue can occur regardless of the type of data destination being used.

    To troubleshoot the issue kindly refer to the documentation for details.

     

    I hope this informatoin helps. Please do let us know if you have any further questions.

     

    Thanks.

    • MOVC's avatar
      MOVC
      Icon for Resolver I rankResolver I
      2 years ago

      Thank you for the reply, but I am able to create new gen 2 dataflows copying data via the gateway to warehouse in new workspaces without issue so I don't believe that this is the problem.

      • MOVC's avatar
        MOVC
        Icon for Resolver I rankResolver I
        2 years ago

        Further testing:

         

        If I create a new Warehouse and a new Dataflow gen2 in the same workspace as the original version, then I get the same error message. However if I use a brand new workspace with the same dataflow query and a new warehouse then it works. 

        Could there be an issue with the Staging Lakehouse in that Workspace?

  • alfBI's avatar
    alfBI
    Icon for Responsive Resident rankResponsive Resident
    2 years ago

    Hi,

     

    We are suffering what seems to be same issue. Our DF2 moving data from on-Premise to a Fabric Warehouse started to fail after having ran for several weeks

     

     

    Error message is shown below

     

     

    We are on lastest version of On-Premise Gateway ad as said, DF2 was working perfectly until last week when it suddenly start to fail.

     

    Regards

     

    Alfons

    • MOVC's avatar
      MOVC
      Icon for Resolver I rankResolver I
      2 years ago

      Unfortunately we were unable to discover the root cause of this issue, but the problem went away after moving from the previous gateway version to the current one (despite the old gateway being able to do the load in a fresh Workspace). 

       

      If your gateway is up to date all I can say is to recreate your dataflow and destination in a fresh workspace and hope for the best.

      • alfBI's avatar
        alfBI
        Icon for Responsive Resident rankResponsive Resident
        2 years ago

        Thx, we thought we were on latest version but it was not the case. After installing latest one works again. As you said a complete mistery why something running after months started to fail.......