Forum Discussion
How to connect to an external API data source that needs ip whitelisting?
- 1 year ago
We too had this issue, and have solved it relatively simply in the end.
The idea is to set up a VNet Data Gateway in Azure, then put it in a subnet fronted with a NAT Gateway which can be configured to have whatever IP address you need.
- Create a VNet Data Gateway: https://learn.microsoft.com/en-us/data-integration/vnet/create-data-gateways
- Create the NAT Gateway and give it a fixed IP address: https://learn.microsoft.com/en-us/azure/nat-gateway/quickstart-create-nat-gateway-portal
- Create a Web/HTTPS activity from Fabric and route it via the VNet Data Gateway:
- 1 year ago
Hi,
Eventually all proposed solutions did not work for us, but we figured it out with a very simple solution, logic app!
what we did:
- Logic app:
- Create a logic app in Azure with a "When an HTTP request is received" trigger . We added a JSON schema for the dynamic parts from our notebook in Fabric (e.g. endpoints and auth token)
- Add HTTP action and built with the dynamic parts from the above trigger
- Add response action.
- Then we whitelisted all outgoing ip adresses from the logic app (found under settings > properties) in the external API
- Lastly, we put the logic app url in our fabric notebook and called it with this line of code:
response = session.post(logic_app_url, headers=headers, data=json.dumps(payload))Our notebook script loops through the API endpoints, calls the Logic App for each one, and writes the results directly to a Delta table and this works perfectly fine!!
Hopefully this helps other people within the (near) future. I struggled with this for a very long time.
- Logic app:
We too had this issue, and have solved it relatively simply in the end.
The idea is to set up a VNet Data Gateway in Azure, then put it in a subnet fronted with a NAT Gateway which can be configured to have whatever IP address you need.
- Create a VNet Data Gateway: https://learn.microsoft.com/en-us/data-integration/vnet/create-data-gateways
- Create the NAT Gateway and give it a fixed IP address: https://learn.microsoft.com/en-us/azure/nat-gateway/quickstart-create-nat-gateway-portal
- Create a Web/HTTPS activity from Fabric and route it via the VNet Data Gateway:
I'm wondering if Step #3 in iainsc 's solution is missing a URL? I tried to find it, but it didn't pop out to me. Hopefully he can provide it.