topic Re: Does ADF Managed VNet with Azure IR Connect to Amazon S3 via Public Internet? in Pipelines

Does ADF Managed VNet with Azure IR Connect to Amazon S3 via Public Internet?

dmct — Wed, 29 Jan 2025 12:22:43 GMT

Hi everyone,

I'm using Azure Data Factory (ADF) with a Managed Virtual Network (VNet) and the Azure Integration Runtime (IR) to connect to Amazon S3. I want to ensure that my data transfer does not traverse the public internet for security reasons.

Could someone please confirm:

If the data transfer between ADF and Amazon S3 goes through the public internet in this setup?
What configurations or settings are required to ensure the data transfer remains within the Microsoft backbone network?
Any best practices or additional considerations to secure the data transfer?

Thank you for your assistance!

Re: Does ADF Managed VNet with Azure IR Connect to Amazon S3 via Public Internet?

nilendraFabric — Wed, 29 Jan 2025 18:06:58 GMT

Hello dmct

enabling Managed VNet alone does not guarantee that traffic to external services like Amazon S3 stays off the public internet.

Quoted from MS doc :

Network security

"By default, ADF transfers data from Amazon S3 to Azure Blob Storage or Azure Data Lake Storage Gen2 using encrypted connection over HTTPS protocol. HTTPS provides data encryption in transit and prevents eavesdropping and man-in-the-middle attacks.

Alternatively, if you don't want data to be transferred over public Internet, you can achieve higher security by transferring data over a private peering link between AWS Direct Connect and Azure Express Route. Refer to the solution architecture in the next section on how this can be achieved."

A great articles covering all aspects of your question:

https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-s3-azure-storage

If this helps please accept the solution.

Thanks

Re: Does ADF Managed VNet with Azure IR Connect to Amazon S3 via Public Internet?

dmct — Thu, 30 Jan 2025 13:01:55 GMT

Thank-you for reply nilendraFabric
That's the article I was looking for and never found it.