Forum Discussion

TK12345's avatar
TK12345
Resolver II
3 months ago

Copy Data with VNET gateway error

Hi all,    I have an issue with my SQL Server / Azure SQL Database extraction. As you can see I connected to my source using a VNET gateway, and I do see the preview data. This table contains 37 ro...
Need Help
pipelines
Show and Tell
v-aatheeque's avatar
v-aatheeque
Community Support
3 months ago

Hi TK12345 
Thanks for reaching out to fabric community forum.

From the screenshot, we can confirm that the source-side connectivity appears to be working correctly because the preview data is loading successfully from the SQL source.

This helps narrow the issue down to the Lakehouse destination/write operation rather than the SQL extraction itself.

Could you please also share a screenshot of the Destination tab configuration from the same Copy Activity?

Additionally, could you help confirm the following:

  • Which Lakehouse is selected as the destination, and how is the connection configured?
  • Are there multiple pipelines or copy activities running in parallel and writing into the same Lakehouse/path simultaneously, especially creating Parquet files?
  • Are you writing into the Tables section or the Files section of the Lakehouse?

These details will help determine whether the issue is related to destination configuration, concurrent write operations or backend behavior.

  • TK12345's avatar
    TK12345
    Resolver II
    3 months ago

    Hi, 

    Thanks for your reply. For the destination I will share the screenshot. FYI, I've tested with a new lakehouse, our normal lakehouse connection, delta, parquet, new schema, dbo, files parquet. Besides that, it is also just a duplicate of another pipeline that is just working fine with the exact same logic. There are no multiple pipelines running, just 1, with 1 table. 

    Preferably I would write this data into tables. But checked all the options, all with same error message.




     

    • v-aatheeque's avatar
      v-aatheeque
      Community Support
      3 months ago

      Hi TK12345 

      Thank you for the clarification.
      Since another pipeline with the same logic is working successfully, this helps narrow the issue specifically to this pipeline/activity configuration or the source table metadata rather than the VNET gateway or Lakehouse itself.

      Could you please try the following:
      Create a new Copy Activity in the same pipeline and configure the source/destination manually instead of duplicating the existing activity.
      Test with a minimal query:
      Then test with:
      SELECT TOP 10 *
      FROM YourTable

      If schema mapping is enabled, please remove and re-import the schema mapping.
      Also verify whether the source table contains unsupported datatypes or special characters in column names.
      This will help determine whether the issue is related to the specific pipeline configuration or the source table metadata.

      • TK12345's avatar
        TK12345
        Resolver II
        3 months ago

        Hi v-aatheeque,
        Thanks for helping. 

        I've tested your suggestions again (tested it before myself), still the same issue. Created a new pipeline just from the UI, new connection, new lakehouse. Just a select limit 10. Still preview data to see, but same error with target.