Forum Discussion

MarcusA's avatar
MarcusA
New Member
2 years ago

DFGen2 Unable to create Destination table in Lakehouse

Hi, trying to create a Destination table in a Lakehouse while using DFGen2 to import data. But I am unable to create tables, both when using Data Destination - Choose destination target dialog, or w...
Dataflows
Yusuf7's avatar
Yusuf7
Frequent Visitor
2 years ago

I've tried the workaround mentioned and it didn't work.

Element115's avatar
Element115
Icon for Memorable Member rankMemorable Member
2 years ago

Screenshot, code and details please.

  • Yusuf7's avatar
    Yusuf7
    Frequent Visitor
    2 years ago

    Thanks for your enquiry. My dataflow Gen2 were working well since I've created them last week. It's only from this morning they started failing to refresh. I detached the destinations and they were working well. But couldn't reconnect with the destination lakehouse again.

     

     

    I had followed the workaround by creatng a staging dataflow and then connect my dataflow gen2 taking data from that staging dataflow. It still didn't work.  

    • Element115's avatar
      Element115
      Icon for Memorable Member rankMemorable Member
      2 years ago

      OK well... if nothing has changed when it stopped working, then it's up to Microsoft to step in here. By the way, we can still see the names you blacked out when we display your screenshot in full screen, the black color is slightly transparent  😉

      • Yusuf7's avatar
        Yusuf7
        Frequent Visitor
        2 years ago

        My bad 😞 Thanks for lettig me know though!!

    • pqian_MSFT's avatar
      pqian_MSFT
      Icon for Microsoft Employee rankMicrosoft Employee
      2 years ago

      This again indicates your GW cannot see the LH server (or cannot connect to it for whatever reason).

      Please gather the following things:

      - Session ID (In the options dialog)

      - Gateway logs with "Additional Logging" turned on while the repro is performed.

       

      P.S., we are working on a change that will show more details when this error happens

      • Yusuf7's avatar
        Yusuf7
        Frequent Visitor
        2 years ago

        My session ID of the failed WriteDataToDestination is is d95209b8-dd1a-439d-9eb9-697cab75c5cf

         

        DataflowID is 795512cf-8e8a-4034-8f29-4641664c8bcb

         

        Tried with the workaround mentioned (staging dataflow extracting data from SharePoint & Second dataflow writing that to destinaion lakehouse), didn't work at all.

         

        The same error not only affected my dataflows writing to the lakehouse/Warehouse, but also occurred with my powerbi connection to dataflow Gen2 & Lakehouse tables.

         

         

        I had just migrated significant portion of my ETL pipelines in fabric dataflow gen2 & Lakehouse, it was working perfectly fine for a week until this suddenly happened, ruining my efforts. 

         

        I'm a business analyst with little understanding of backend security protocols & gateways, therefore, it'd be helpful to have a clearer solution.