Not able to write to datalake from virtual data gateway with Copy Data Task in Fabric

[kjetil_hamre]

Loading with dataflow2 pipelines works like a charm. However, when I try to do the same with copy-task in a datafactory pipeline (my preferred method) or a copy task - it just hangs for hours, doing nothing but exhausting my fabric capacity.

In the data factory pipeline, I am able preview the data in the source preview box. I am also able to write from one lakehouse table to another. I can also preview source in mapping.

[ex_kjetilh]

It is now hanging like this on a table with 17 rows..

[AntoineW]

Hello @kjetil_hamre,

Why does Copy Data task hang when writing to OneLake via Virtual Network Data Gateway?

• Root Cause:
  Copy Activity in Fabric pipelines can stall if:
  • The VNet Data Gateway is misconfigured or lacks outbound connectivity.
  • The Lakehouse sink settings or schema are invalid.
  • The scenario is not fully supported for Copy Activity (especially with large data or transformations).
  • Capacity is consumed but the write phase is blocked.

---

Key Points

1. Copy Activity relies on the gateway and Fabric connectors, so any network or schema issue can cause it to hang.
2. Capacity exhaustion happens because compute is allocated even if the write is stuck.

---

✅ How to fix it

• Check gateway configuration: Ensure VNet Data Gateway is healthy, linked to the right capacity, and has outbound HTTPS (443) access to Fabric endpoints.
• Validate Lakehouse sink: Confirm the destination table or folder exists and schema matches.
• Update pipeline settings:
  • Reduce parallelism or disable unnecessary staging.
  • If possible, enable Fast Copy (or switch to Dataflow Gen2 for heavy loads).
• Monitor capacity: Use the Fabric Capacity Metrics app to avoid CU waste.
• Permissions: Make sure the pipeline identity or service principal has Contributor access to the Lakehouse workspace.

Source : 

- https://learn.microsoft.com/en-us/fabric/data-factory/dataflows-gen2-fast-copy

- https://learn.microsoft.com/en-us/fabric/data-factory/copy-job-with-virtual-network-data-gateway

- https://learn.microsoft.com/en-us/data-integration/vnet/use-data-gateways-pipeline-fabric

- https://learn.microsoft.com/en-us/fabric/data-factory/gateway-considerations-output-destinations

- https://learn.microsoft.com/en-us/fabric/fundamentals/decision-guide-pipeline-dataflow-spark

Hope it can help you ! 

Best regards,

Antoine

[kjetil_hamre]

Thank you for very prompt answer. I think it must be the VNET. Investigating.

[v-veshwara-msft]

Hi @kjetil_hamre ,
Thanks for reaching out to Microsoft Fabric Community and thanks for the update.

Good to know you are investigating the VNET side.

As @AntoineW explained, Copy Activity in Fabric pipelines depends on the Virtual Network Data Gateway for outbound connectivity to Fabric endpoints. If the gateway is not correctly configured or restricted on outbound HTTPS (443), the copy task may hang while still consuming capacity. It is also important to make sure the Lakehouse sink is valid, with schema alignment and permissions in place, otherwise the write phase can remain blocked even though preview works.

There have been similar reports where writes to Lakehouse failed due to routing. One case was resolved by adding a microsoft.storage service endpoint to the delegated gateway subnet, details here: Solved: Re: VNET data gateway cannot write to lakehouse - Microsoft Fabric Community 

Microsoft also announced GA support for Virtual Network Data Gateway with Fabric Pipeline, Dataflow Gen2 Fast Copy, and Copy Job, which may improve reliability in such cases.

Virtual Network Data Gateway Support for Fabric Pipeline, Dataflow Gen2 Fast Copy, and Copy Job (Gen...

As an alternative, you could also test using the on-premises data gateway, which is supported in Fabric pipelines and Dataflow Gen2. 

How to access on-premises data sources in Data Factory - Microsoft Fabric | Microsoft Learn

Hope this helps. Please reach out for further assistance.

Thank you.

[ex_kjetilh]

Thank you. We have found some issues with the virtual gateway and DNS, and got it to work for a brief period, but now not working again. Still investigating.