topic Pull data from On-Prem sql to ADLS in parquet using ADF pipeline copy activity performance issue in Pipelines

Pull data from On-Prem sql to ADLS in parquet using ADF pipeline copy activity performance issue

hetadesai_56 — Mon, 03 Mar 2025 13:27:39 GMT

I tried to pull large volume of data from On-Prem sql server to ADLS in parquet format. Data volume is aroud 300 GB. Pulled data using static partitioning on Date. The pipeline execution time is around 19 hours. Please provide alternative solution or performance improvement srtatef

Re: Pull data from On-Prem sql to ADLS in parquet using ADF pipeline copy activity performance issue

NandanHegde — Mon, 03 Mar 2025 15:05:31 GMT

please refer the below MSFT Doc:

https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-performance-features

ADF and Fabric data pipeline follow the same principles

Re: Pull data from On-Prem sql to ADLS in parquet using ADF pipeline copy activity performance issue

Anonymous — Mon, 21 Apr 2025 04:04:57 GMT

Hi hetadesai_56,

Thank you for reaching out in Microsoft Community Forum.

Thank you NandanHegde for the helpful response.

As suggested by NandanHegde, I hope this information was helpful. Please let me know if you have any further questions or you'd like to discuss this further. If this answers your question, please "Accept as Solution" and give it a 'Kudos' so others can find it easily.

Please continue using Microsoft community forum.

Regards,
Pavan.

Re: Pull data from On-Prem sql to ADLS in parquet using ADF pipeline copy activity performance issue

Anonymous — Fri, 25 Apr 2025 08:54:09 GMT

Hi hetadesai_56,

I hope this information is helpful. Please let me know if you have any further questions or if you'd like to discuss this further. If this answers your question, kindly "Accept as Solution" and give it a 'Kudos' so others can find it easily.

Thank you,
Pavan.

Re: Pull data from On-Prem sql to ADLS in parquet using ADF pipeline copy activity performance issue

Anonymous — Tue, 29 Apr 2025 04:37:37 GMT

Hi hetadesai_56,

I wanted to follow up since we haven't heard back from you regarding our last response. We hope your issue has been resolved.
If the community member's answer your query, please mark it as "Accept as Solution" and select "Yes" if it was helpful.
If you need any further assistance, feel free to reach out.

Please continue using Microsoft community forum.

Thank you,
Pavan.