Forum Discussion

yjh's avatar
yjh
Helper II
2 years ago
Solved

Iirresponsible?

Many people have said that DataFlow gen2 writing to WareHouse from various sources will continue abnormally for many hours with a high probability of failure, why can't Microsoft solve this problem f...
Dataflows
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi yjh 
    Thanks for using Fabric Community. 
    Apologies for the issue you have been facing. We understand that this issue has caused problems to your operations, and we sincerely apologize for the inconvenience. We want to assure you that Microsoft is taking this issue very seriously and is actively working on a fix. Our engineering teams have identified the root cause of the problem and are currently developing a solution. 

     

    This is a known issue and the team is aware about this. But we do not have any ETA regarding the fix. We will continue to keep you updated on the progress of the fix and provide you with additional information as it becomes available. We appreciate your patience and understanding as we work to resolve this issue.

     

    Thank you.

Joshrodgers123's avatar
Joshrodgers123
Advocate V
2 years ago

Hi Anonymous - if this is a known issue, why hasn't it been added to the known issues list? It seems many customers are having similar issues. According to the GA status of Fabric, we expect these things will work and there is no documentation or posted known issue that says otherwise.

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi Joshrodgers123 
      Thanks for letting me know. I have informed the same to the internal team. 
      I will get back to you once I hear back from them. 
      Appreciate your patience.

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi Joshrodgers123 
      Just as a work around, instead of loading on prem data to Dataflow Gen2 and then to warehouse (On-Prem -> DFg2 -> Warehouse) can you please try to test by loading the data from on-prem to Dataflow Gen2 and then to Lakehouse.(On-Prem -> DFg2 -> Lakehouse) (files or tables, would test both)
      Then create a Data Pipeline to move data from Lakehouse to Warehouse.

      Just like the medallion architecture we see often, we want to do a quick load of the source into a Raw layer, then work our way to the Warehouse which would be Silver/Gold. 


      The issue which you are facing while loading the data into warehouse is just temporary. This won't be a thing once Data Pipelines can use On Prem Data Gateway which is in development.


      Hope this helps. Please let me know if you have any further questions.