Enabling Azure Data Factory to Access Restricted REST API and Transfer Data to Lakehouse: IP Address

[Anonymous]

How can I configure Azure Data Factory to access a REST API and transfer data to a Lakehouse while considering restrictions on IP addresses or domain names? Specifically, I need to allow Azure Data Factory to access the REST API within Fabric, but I have limitations in terms of IP addresses or domain name whitelisting. What steps should I follow to enable this connectivity and ensure data transfer to the Lakehouse? I would appreciate detailed guidance and any specific configurations or settings that need to be applied within Azure Data Factory and the REST API's security settings.

[ajarora]

Couple of more questions:

1. Do you specifically mean Azure Data Factory or Fabric Data Factory ? The former doesnt support reading/writing into Fabric Lakehouse currently.

2. What is the Fabric API you plan to call for accessing data ?

[Anonymous]

1. Fabric Data Factory. Based on the answer provided by @Abhishek_Narain, the access to the REST API was successful.

[ajarora]

Curious which API are you looking to call ?

[Anonymous]

Tripadvisor Content API

[Abhishek_Narain · Accepted solution]

Currently, you can allow-list the whole Azure Data Factory IP range at your source that hosts the REST API. The IP range is shared across Fabric Data Factory. Do discuss this approach with your security team. 
https://learn.microsoft.com/en-us/azure/data-factory/azure-integration-runtime-ip-addresses#azure-in... (Use the integration runtime IP address). If you want a reduced IP ranges then allow-list only a specific region that your Fabric tenant is setup in. 

Alternatively, we are working on adding On-premises gateway support for pipelines in Fabric which will allow you to read data from private rest api sources from within your network or help you with a static IP (on-premises gateway node).

[DavidCousins]

Nice, thanks for that.

On the reverse, is there anything planned to allow us to control what Data Factory can access? Not sure I'm a fan of allowing it to access any web API on the public internet, especially as it can use them as a data sink.

I popped an idea in here: Restrict web access for Data Factory 

[Anonymous]

How can I integrate a web API for data import into ADF after the recent changes? Could you please advise me on the steps to follow?

[Anonymous]

@DavidCousins @ajarora What is the idea of having the option in ADF to create web connections and use them as a source in copy pipeline activities since web APIs are no longer allowed?

[GraceGu]

@Abhishek_Narain to comment.