Data copy from on premises to Fabric SQL database fails due to lost connectivity Fabric SQL db

[MarkLobbezoo89]

Hi, 

doubting where to put this issue, Data pipeline or Fabric SQL database. 

We have a pipeline running now for 3 months without any issues copying 150 tables from a on premises SQL database to a Fabric SQL database. Since last Saturday the pipeline fails due to a lost connectivity with the Fabric SQL database. 

A couple of tables are copied succesfully and then the pipeline starts failing giving the following error: 

ErrorCode=SqlFailedToConnect,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Cannot connect to SQL Database. Please contact SQL server team for further support. Server: 'xx.database.fabric.microsoft.com,1433', Database: 'fsql_xx', User: ''. Check the connection configuration is correct, and make sure the SQL Database firewall allows the Data Factory runtime to access.,Source=Microsoft.DataTransfer.Connectors.MSSQL,''Type=Microsoft.Data.SqlClient.SqlException,Message=Login failed for user '<token-identified principal>'.Reason: Validation of user's permissions failed. Verify the user has the Read item permission. For more information, see https://go.microsoft.com/fwlink/?linkid=2281595.,Source=Framework Microsoft SqlClient Data Provider,'

Any idea's?

We are running on a F16 capacity.

There are 2 tables that were normally running about an hour, quite a lot of rows to copy. Is there somesort of treshold? 

In the admin report we also see that since Saturday the tag of the run is changed from non-billable to billable...

Regards,

Mark  

[Anonymous · Accepted solution]

Hi @MarkLobbezoo89

Thank you very much Srisakthi for your prompt reply.

It looks like you have identified the issue as capacity being exceeded, causing throttling in the Fabric Capacity Metrics report. Based on the graph you provided, there is a clear increase in the Interactive Latency percentage starting on February 7, 2025. There were indeed some updates during that time period.

According to the Microsoft Fabric Updates blog, several new features and improvements were introduced in early February 2025. These updates may impact the management and use of capacity.

Microsoft Fabric January 2025 update | Microsoft Fabric Blog | Microsoft Fabric

You may consider increasing your capacity SKU to handle your workload more efficiently. If your current F16 capacity is insufficient for your needs, you can upgrade to a higher capacity. 

Or try to spread your workloads around to avoid peak times and reduce the likelihood of throttling. Adjust your configuration and workload management strategies based on the updates. 

Regards, 

Nono Chen 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly. 

[Srisakthi]

Hi @MarkLobbezoo89 ,

Can you check what are all the Fabric items were running at that time. The image says it is for the entire time period, so there could be anyother Fabric items which is consuming Capacity. So better check what are all other Fabric items consuming capacity at that time.

Regards,

Srisakthi

[MarkLobbezoo89]

Found the issue. We are overreaching our capacity. 
In the past this was not an issue but as from 7 FEB 2025 aroud 14:30 (UTC + 1), we seeing Trottling  in the capacity metric report: 

Anyone a clue if there was an update of Fabric around that time? The workload on our side hasn't changed. 

Regards,

Mark 

[Srisakthi]

Hi @MarkLobbezoo89 ,

As per your error message it says that connection issue.

Please check the below

1. if there is any change in connection string, user name and password.

2. If the sql is put under Virtual Network. If it is under virtual network then need to whitelist the Pipeline IP

Regards,

Srisakthi