topic Re: The copy from sql server on premise has been released but it doesn't work in Pipelines

The copy from sql server on premise has been released but it doesn't work

marcoG — Sun, 24 Mar 2024 11:17:19 GMT

On-premise SQL Server copy task remains in queued state indefinitely

Re: The copy from sql server on premise has been released but it doesn't work

v-cboorla-msft — Mon, 25 Mar 2024 08:19:46 GMT

Thanks for using Microsoft Fabric Community.

As I understand you are facing an issue with an on-premise SQL Server copy task getting stuck in the queued state within Microsoft Fabric.

Could you please share the screenshot of the error message you are seeing or some additional information about the activity that you are using in the Microsoft Fabric portal for the queued task?

Thanks.

Re: The copy from sql server on premise has been released but it doesn't work

marcoG — Mon, 25 Mar 2024 08:25:34 GMT

I dont get any errors... it stays tha way:

Re: The copy from sql server on premise has been released but it doesn't work

marcoG — Mon, 25 Mar 2024 10:24:38 GMT

The connections test are ok...

Re: The copy from sql server on premise has been released but it doesn't work

v-cboorla-msft — Mon, 25 Mar 2024 10:07:54 GMT

Hi marcoG

I successfully replicated the reported scenario and was able to run the copy activity without encountering any difficulties. For your reference, I have attached the screenshots.

It appears to be an intermittent issue with the data pipeline run. To ensure reliable data processing, I recommend you to attempt the activity again after a brief period.

Try to delete the activity, refresh the browser, add the activity again, and run the pipeline, this might help you.

Try to retry a data pipeline run this might help you, for further details on retrying data pipeline runs, kindly refer to retry a data pipeline run.

If the issue still persists please do let us know.

I hope this information helps.

Thanks.

Re: The copy from sql server on premise has been released but it doesn't work

marcoG — Mon, 25 Mar 2024 10:31:12 GMT

Have you used the gateway too?

Re: The copy from sql server on premise has been released but it doesn't work

v-cboorla-msft — Wed, 27 Mar 2024 11:24:38 GMT

Hi marcoG

Following up to see if you have a resolution yet or still facing the issue. In case if you have any resolution please do share that same with the community as it can be helpful to others.
If you have any question relating to the current thread, please do let us know and we will try out best to help you.
In case if you have any other question on a different issue, we request you to open a new thread.

If the issue still persists, please refer : Set new firewall rules on server running the gateway

For more details please refer : Currently supported monthly updates to the on-premises data gateways

I hope this information helps.

Thanks.

Re: The copy from sql server on premise has been released but it doesn't work

marcoG — Wed, 27 Mar 2024 11:31:36 GMT

it still does not work

Re: The copy from sql server on premise has been released but it doesn't work

v-cboorla-msft — Wed, 27 Mar 2024 11:36:09 GMT

Hi marcoG

Could you please share me the screenshots of the error that you are getting along with the copy activity details. This will help me in understanding the issue and help you in better way.

Thanks.

Re: The copy from sql server on premise has been released but it doesn't work

marcoG — Wed, 27 Mar 2024 11:38:53 GMT

As I wrote previously I don't get any errors... it remains in the queued state indefinitely

Re: The copy from sql server on premise has been released but it doesn't work

v-cboorla-msft — Wed, 27 Mar 2024 12:21:43 GMT

Hi marcoG

Monitor Pipeline Runs: Use the Azure Data Factory monitoring tools to view activity logs. These logs might provide detailed information about the issue, pinpointing specific errors or bottlenecks causing the queueing.

For details please refer : Monitor data pipeline runs.

Thanks.

Re: The copy from sql server on premise has been released but it doesn't work

marcoG — Wed, 27 Mar 2024 12:54:17 GMT

Thank you for your general answers.
If I have a pipeline with only 1 activity that copies a table of 2 rows I cannot take an infinite amount of time to complete, do you agree?
So displaying the monitoring hub which tells me that the task is in progress, what do I need it for?

Re: The copy from sql server on premise has been released but it doesn't work

v-cboorla-msft — Wed, 27 Mar 2024 15:05:08 GMT

Hi marcoG

Apologies for the issue you have been facing.

It's difficult to tell what could be the reason for this performance. Please reach out to our support team so they can do a more thorough investigation on why this it is happening: Link

After creating a Support ticket please provide the ticket number as it would help us to track for more information.

Thanks.

Re: The copy from sql server on premise has been released but it doesn't work

Anonymous — Wed, 27 Mar 2024 15:14:34 GMT

I am experiencing the exact same issue.

As suggested, I deleted my "Copy Data" activity and recreated it but the same issue occurred. I also tried re creating the Copy Data activity using the "Copy Assistant", but again I got the same result.

Any help would be greatly appreciated.

Re: The copy from sql server on premise has been released but it doesn't work

Anonymous — Thu, 28 Mar 2024 11:55:26 GMT

Hi v-cboorla-msft ,

I have opened a support ticket regarding this issue: 2403280040003647.

Please let me know if you need any other details.

Thanks

Re: The copy from sql server on premise has been released but it doesn't work

v-cboorla-msft — Thu, 28 Mar 2024 11:55:39 GMT

Hi Anonymous

In order to understand your specific data copy scenario, Could you please share some more details about your copy data activity. This will help me to understand the query and guide in better way. If possible, could you provide information on the following:

Source and Sink Details: Can you elaborate on the types of data stores involved in the copy operation?

Data Transformation Specifications: Are there any transformations being applied to the data during the copy process?

Screenshots: If feasible, sharing screenshots of the source and sink configuration within your data copy activity would be highly beneficial for visualization purposes.

Thanks.

Re: The copy from sql server on premise has been released but it doesn't work

Anonymous — Thu, 28 Mar 2024 12:03:47 GMT

Aboslutely! The Copy Data will copy data from an On-Prem SQL Server hosted on AWS on an EC2 and Copy this data to a Lakehouse in the same workspace as the Pipeline. The source is a SQL Server 2019 standard.

There are no transformation to the data at all; the purpose is to do a strait copy of the data from On-Prem to the Fabric Lakehouse.

Here is the details of the source:

Here are the details of the sink:

Here are the mapping configs:

And settings are defaults:

Let me know if you need anythign else.

Re: The copy from sql server on premise has been released but it doesn't work

marcoG — Tue, 02 Apr 2024 11:58:35 GMT

is there any news?

Re: The copy from sql server on premise has been released but it doesn't work

Anonymous — Tue, 02 Apr 2024 12:21:05 GMT

Hey MarcoG,

No news yet from Microsoft. I have provided them (MS) with some details about the pipeline and am waiting to hear back.

I will let you know as soon as I have any infos.

Cheers

JFG

Re: The copy from sql server on premise has been released but it doesn't work

re84uk — Wed, 10 Apr 2024 15:15:30 GMT

I have the same problem, attempting to copy data from on-prem SQL Server to lakehouse via data pipelines just sits indefinitely at status "In Progress".