Forum Discussion
How to connect to an external API data source that needs ip whitelisting?
I'm trying to connect to an external API that requires IP whitelisting, but its challenging because Fabric uses a range of IP addresses that change dynamically. Its not an option to whitelist the whole range of ip adresses.
Im currently thinking of setting up an Azure VM with a static public IP and use a private endpoint between the Azure VM and Fabric.
Before proceeding, I’d love to know if anyone has done a similar setup or is there a better alternative for this scenario?
Appreciate any insights—thanks in advance!
We too had this issue, and have solved it relatively simply in the end.
The idea is to set up a VNet Data Gateway in Azure, then put it in a subnet fronted with a NAT Gateway which can be configured to have whatever IP address you need.
- Create a VNet Data Gateway: https://learn.microsoft.com/en-us/data-integration/vnet/create-data-gateways
- Create the NAT Gateway and give it a fixed IP address: https://learn.microsoft.com/en-us/azure/nat-gateway/quickstart-create-nat-gateway-portal
- Create a Web/HTTPS activity from Fabric and route it via the VNet Data Gateway:
Hi,
Eventually all proposed solutions did not work for us, but we figured it out with a very simple solution, logic app!
what we did:
- Logic app:
- Create a logic app in Azure with a "When an HTTP request is received" trigger . We added a JSON schema for the dynamic parts from our notebook in Fabric (e.g. endpoints and auth token)
- Add HTTP action and built with the dynamic parts from the above trigger
- Add response action.
- Then we whitelisted all outgoing ip adresses from the logic app (found under settings > properties) in the external API
- Lastly, we put the logic app url in our fabric notebook and called it with this line of code:
response = session.post(logic_app_url, headers=headers, data=json.dumps(payload))Our notebook script loops through the API endpoints, calls the Logic App for each one, and writes the results directly to a Delta table and this works perfectly fine!!
Hopefully this helps other people within the (near) future. I struggled with this for a very long time.
- Logic app:
8 Replies
- nilendraFabric
Super User
Hello MartijnCyg
this approach is simple and future proofDeploy an Azure Function with a static outbound IP to act as an intermediary between Fabric and the API:
• The Function handles authentication/requests
• Whitelist the Function’s static IP at the API endpoint
• Fabric notebooks call the Function instead of direct API accessPlease accept this solution and give kudos if this is helpful
Hi MartijnCyg,
Thanks nilendraFabric for addressing the issue.
we would like to follow up to see if the solution provided by the super user resolved your issue. Please let us know if you need any further assistance.
If our super user response resolved your issue, please mark it as "Accept as solution" and click "Yes" if you found it helpful.Regards,
Vinay PabbuHi Anonymous , nilendraFabric ,
Thank you for your response. I never created a function before, but when trying to deploy the azure function, but my function keeps dissapearing and im not able to create another as my "add function button" is gone...
I do see the app files, and the app service is also working.
I deleted the function and added it again in the portal, but same thing keeps happening. I do have suffiencient rights.
What am i doing wrong?
Hi MartijnCyg,
Since your request is related to Azure. I suggest you to raise it in Azure community forum. So that your issue will be addressed more effectively.
Category: Azure | Microsoft Community Hub
Regards,
Vinay Pabbu
We too had this issue, and have solved it relatively simply in the end.
The idea is to set up a VNet Data Gateway in Azure, then put it in a subnet fronted with a NAT Gateway which can be configured to have whatever IP address you need.
- Create a VNet Data Gateway: https://learn.microsoft.com/en-us/data-integration/vnet/create-data-gateways
- Create the NAT Gateway and give it a fixed IP address: https://learn.microsoft.com/en-us/azure/nat-gateway/quickstart-create-nat-gateway-portal
- Create a Web/HTTPS activity from Fabric and route it via the VNet Data Gateway:
I'm wondering if Step #3 in iainsc 's solution is missing a URL? I tried to find it, but it didn't pop out to me. Hopefully he can provide it.
Hi,
Eventually all proposed solutions did not work for us, but we figured it out with a very simple solution, logic app!
what we did:
- Logic app:
- Create a logic app in Azure with a "When an HTTP request is received" trigger . We added a JSON schema for the dynamic parts from our notebook in Fabric (e.g. endpoints and auth token)
- Add HTTP action and built with the dynamic parts from the above trigger
- Add response action.
- Then we whitelisted all outgoing ip adresses from the logic app (found under settings > properties) in the external API
- Lastly, we put the logic app url in our fabric notebook and called it with this line of code:
response = session.post(logic_app_url, headers=headers, data=json.dumps(payload))Our notebook script loops through the API endpoints, calls the Logic App for each one, and writes the results directly to a Delta table and this works perfectly fine!!
Hopefully this helps other people within the (near) future. I struggled with this for a very long time.
- Logic app:
