How to connect to an external API data source that needs ip whitelisting?

[MartijnCyg]

I'm trying to connect to an external API that requires IP whitelisting, but its challenging because Fabric uses a range of IP addresses that change dynamically. Its not an option to whitelist the whole range of ip adresses.

Im currently thinking of setting up an Azure VM with a static public IP and use a private endpoint between the Azure VM and Fabric.

Before proceeding, I’d love to know if anyone has done a similar setup or is there a better alternative for this scenario? 

Appreciate any insights—thanks in advance!

[iainsc · Accepted solution]

We too had this issue, and have solved it relatively simply in the end.

The idea is to set up a VNet Data Gateway in Azure, then put it in a subnet fronted with a NAT Gateway which can be configured to have whatever IP address you need.

1. Create a VNet Data Gateway: https://learn.microsoft.com/en-us/data-integration/vnet/create-data-gateways
2. Create the NAT Gateway and give it a fixed IP address: https://learn.microsoft.com/en-us/azure/nat-gateway/quickstart-create-nat-gateway-portal
3. Create a Web/HTTPS activity from Fabric and route it via the VNet Data Gateway: 

[iainsc · Accepted solution]

We too had this issue, and have solved it relatively simply in the end.

The idea is to set up a VNet Data Gateway in Azure, then put it in a subnet fronted with a NAT Gateway which can be configured to have whatever IP address you need.

1. Create a VNet Data Gateway: https://learn.microsoft.com/en-us/data-integration/vnet/create-data-gateways
2. Create the NAT Gateway and give it a fixed IP address: https://learn.microsoft.com/en-us/azure/nat-gateway/quickstart-create-nat-gateway-portal
3. Create a Web/HTTPS activity from Fabric and route it via the VNet Data Gateway: 

[MartijnCyg · Accepted solution]

Hi, 

Eventually all proposed solutions did not work for us, but we figured it out with a very simple solution, logic app!

what we did:

1. Logic app:
   • Create a logic app in Azure with a "When an HTTP request is received" trigger . We added a JSON schema for the dynamic parts from our notebook in Fabric (e.g. endpoints and auth token)
   • Add HTTP action and built with the dynamic parts from the above trigger
   • Add response action.
2. Then we whitelisted all outgoing ip adresses from the logic app (found under settings > properties) in the external API 
3. Lastly, we put the logic app url in our fabric notebook and called it with this line of code:

 response = session.post(logic_app_url, headers=headers, data=json.dumps(payload))

Our notebook script loops through the API endpoints, calls the Logic App for each one, and writes the results directly to a Delta table and this works perfectly fine!!

Hopefully this helps other people within the (near) future. I struggled with this for a very long time. 

[scott_stauffer1]

I'm wondering if Step #3 in @iainsc 's solution is missing a URL?  I tried to find it, but it didn't pop out to me.  Hopefully he can provide it. 

[v-vpabbu]

Hi @MartijnCyg,

Thanks @nilendraFabric  for addressing the issue.

we would like to follow up to see if the solution provided by the super user resolved your issue. Please let us know if you need any further assistance.
If our super user response resolved your issue, please mark it as "Accept as solution" and click "Yes" if you found it helpful.

Regards,
Vinay Pabbu

[MartijnCyg]

Hi @v-vpabbu , @nilendraFabric ,

Thank you for your response. I never created a function before, but when trying to deploy the azure function, but my function keeps dissapearing and im not able to create another as my "add function button" is gone...

I do see the app files, and the app service is also working.

 

I deleted the function and added it again in the portal, but same thing keeps happening. I do have suffiencient rights.

What am i doing wrong?

[v-vpabbu]

Hi @MartijnCyg,

Since your request is related to Azure. I suggest you to raise it in Azure community forum. So that your issue will be addressed more effectively.

Category: Azure | Microsoft Community Hub 

Regards,

Vinay Pabbu

[v-vpabbu]

Hi @MartijnCyg,

As we haven't heard back from you, I hope you have raised in Azure community. At this time we are closing this thread. If you have any further issues, please start a new thread in the community forum, and we are here to assist you. Thankyou for your understanding and continuous support.

Thank you for being part of the Microsoft Fabric Community.

Regards,

Vinay Pabbu

[nilendraFabric]

Hello @MartijnCyg 

this approach is simple and future proof

Deploy an Azure Function with a static outbound IP to act as an intermediary between Fabric and the API:
• The Function handles authentication/requests
• Whitelist the Function’s static IP at the API endpoint
• Fabric notebooks call the Function instead of direct API access

https://stackoverflow.com/questions/79155429/azure-api-management-access-from-microsoft-fabric-handl...

Please accept this solution and give kudos if this is helpful