SFTP copy job failure - Data gateway issue

[cheryl0316]

I have created a data pipeline for a copy job and would like to get data from SFTP to Fabric. The flow is:
Azure Blob → SFTP → Fabric

Using the same server, username, and password, it used to work when using the on-premises data gateway. Since the database has been moved to the cloud, I can no longer use the on-premises data gateway.

Instead, I have to choose Data Gateway (none) which causes the copy job to fail. It shows an error: “Unable to list files.”

If I go to “Manage Connections and Gateways,” the connection shows as online.

Is there any way to resolve this without using the on-premises data gateway?

Thank you.

[v-dineshya]

Hi @cheryl0316 ,

Thank you for reaching out to the Microsoft Community Forum.

Hi  @arabalca  and @tayloramy ,Thank you for your prompt responses.

Hi @cheryl0316  ,  could you please try the proposed solutions shared by  @arabalca  and @tayloramy   ? Let us know if you’re still facing the same issue we’ll be happy to assist you further.

Regards,

Dinesh

[tayloramy · Accepted solution]

Hi @cheryl0316, 

The error "The remote name could not be resolved:xxxxxxx.blob.core.windows.net" tells is that either your storage account is restructed from public networks and Fabric can't access it, you have the wrong storage name, or you don't have permissions. 

I would recommend using a Lakehouse Shortcut to the blob storage so you can access it directly in Fabric without needing to copy it. 

Create an Azure Blob Storage shortcut - Microsoft Fabric | Microsoft Learn

The best way to authenciate with this is to create a workspace identity: Workspace identity - Microsoft Fabric | Microsoft Learn

and then once created, grant that service principal access to the blob storage. 

[cheryl0316]

Hi @tayloramy 

Yes the storage account is restricted from public access. We allow Fabric to go through the firewall and it works. Thank you.

[arabalca]

Hi @cheryl0316 
my first question would be whether there is any explanation for that blob → sftp → fabric traceability.
From my perspective, I would suggest simplifying the architecture by using a direct Fabric shortcut to Data Lake / Blob Storage. This would allow you to avoid unnecessary data movement and eliminate the need for intermediate pipelines, making the solution more efficient and easier to maintain.
Regarding the gateway issue, this kind of error is often caused by an outdated version.
If the gateway has been installed for more than 6 months, I would recommend reinstalling or updating it to the latest version, as this usually resolves many connectivity issues.
Please check this and let me know 👍
If my answer helped you, I would really appreciate it if you could like the comment and mark it as the accepted solution, as it may help others facing the same issue.
Thanks a lot 🙌

[cheryl0316]

Hi @arabalca thanks for your reply
I just created a copy job using Azure Blob but it shows an error: An exception occurred: DataSource.Error: AzureBlobs failed to get the response: 'The remote name could not be resolved:xxxxxxx.blob.core.windows.net

I think I entered the right account name and key, but I still can't access it. Do you think it could be related to the security setting?

Is data gateway (none) = cloud connection? If so we don't need to install gateway?

[arabalca]

Hi @cheryl0316 

The gateway is used as a secure bridge between on-premise environments (such as your SFTP) and the cloud, in this case Fabric.

On the other hand, Blob storage is already in the cloud, so no gateway is required to access it.

Here’s a tutorial about shortcuts in Fabric where you can check it in detail:
👉 https://learn.microsoft.com/en-us/fabric/onelake/create-blob-shortcut

If my answer helped you, I would really appreciate a 👍 and if you could mark it as the accepted solution. This helps other community members and also motivates me to keep contributing. Thanks a lot!