Forum Discussion
Enabling Azure Data Factory to Access Restricted REST API and Transfer Data to Lakehouse: IP Address
How can I configure Azure Data Factory to access a REST API and transfer data to a Lakehouse while considering restrictions on IP addresses or domain names? Specifically, I need to allow Azure Data Factory to access the REST API within Fabric, but I have limitations in terms of IP addresses or domain name whitelisting. What steps should I follow to enable this connectivity and ensure data transfer to the Lakehouse? I would appreciate detailed guidance and any specific configurations or settings that need to be applied within Azure Data Factory and the REST API's security settings.
Currently, you can allow-list the whole Azure Data Factory IP range at your source that hosts the REST API. The IP range is shared across Fabric Data Factory. Do discuss this approach with your security team.
https://learn.microsoft.com/en-us/azure/data-factory/azure-integration-runtime-ip-addresses#azure-integration-runtime-ip-addresses-specific-regions (Use the integration runtime IP address). If you want a reduced IP ranges then allow-list only a specific region that your Fabric tenant is setup in.
Alternatively, we are working on adding On-premises gateway support for pipelines in Fabric which will allow you to read data from private rest api sources from within your network or help you with a static IP (on-premises gateway node).
9 Replies
Abhishek_Narain to comment.
Currently, you can allow-list the whole Azure Data Factory IP range at your source that hosts the REST API. The IP range is shared across Fabric Data Factory. Do discuss this approach with your security team.
https://learn.microsoft.com/en-us/azure/data-factory/azure-integration-runtime-ip-addresses#azure-integration-runtime-ip-addresses-specific-regions (Use the integration runtime IP address). If you want a reduced IP ranges then allow-list only a specific region that your Fabric tenant is setup in.
Alternatively, we are working on adding On-premises gateway support for pipelines in Fabric which will allow you to read data from private rest api sources from within your network or help you with a static IP (on-premises gateway node).Nice, thanks for that.
On the reverse, is there anything planned to allow us to control what Data Factory can access? Not sure I'm a fan of allowing it to access any web API on the public internet, especially as it can use them as a data sink.
I popped an idea in here: Restrict web access for Data Factory
Couple of more questions:
1. Do you specifically mean Azure Data Factory or Fabric Data Factory ? The former doesnt support reading/writing into Fabric Lakehouse currently.
2. What is the Fabric API you plan to call for accessing data ?
1. Fabric Data Factory. Based on the answer provided by Abhishek_Narain, the access to the REST API was successful.
Curious which API are you looking to call ?
