Forum Discussion

prom's avatar
prom
Frequent Visitor
2 years ago

Dataflow Gen2 from on prem Oracle - burning CU like crazy

Hello @all   We need to transfer data from on-prem Oracle table to the Lakehouse. As of tuday, in Fabric, is it only possible using DF Gen2 (copy pipeline doesn't support on-prem gw yet).    We a...
Dataflows
General Comment
General Question
Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi prom ,

Thanks for using Fabric Community.
Unfortunately you can only connect to On-prem is via DataFlow Gen 2.
The feature - "Connecting to the On-premise gateways using Pipelines" is still on roadmap. I will keep you posted regarding the updates.

 

Docs to refer -
What's new and planned for Data Factory in Microsoft Fabric - Microsoft Fabric | Microsoft Learn
How to access on-premises data sources in Data Factory - Microsoft Fabric | Microsoft Learn

Hope this helps. Please let me know if you have any further questions.

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hello prom ,

    We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
    Otherwise, will respond back with the more details and we will try to help .

    • prom's avatar
      prom
      Frequent Visitor
      2 years ago

      Hello,

       

      Thanks for quick answer. I'll probably wait for migration CP pipelines to Fabric. I'll try using ADF or custom spark cluster as a workaround.

       

      Best Regards

      • pqian_MSFT's avatar
        pqian_MSFT
        Icon for Microsoft Employee rankMicrosoft Employee
        2 years ago

        prom one thing you could try is disable staging for your entities, it should at least complete without running twice.

         

        We are aware of an issue that may slow down ingestion into LH if your data contains large strings (>4k in a cell) or column that is primarily nulls. If that is what you are running into, then the next GW patch will have this fix and it should speed it up by another 40%.