A reference architecture for private API access from Microsoft Fabric

[dimkalamaras]

Fabric notebooks often need to read from authenticated REST APIs that live inside a network boundary — historians, MES, custom microservices, on-prem inventory systems. Public IPs are off the table, Fabric can't host a VPN client, and copying the data into the lakehouse first defeats the point.

This repo is a reference architecture for the path that actually works in production: Fabric → Managed Private Endpoint → Azure Function App (acting as a thin reverse proxy) → N private backends — with no changes to the backend code.

The shape

The Function App is a ~100-line Python proxy on Flex Consumption, VNet-integrated:

• One catch-all route /<backend>/<anything> — the first URL segment selects the backend, everything after is forwarded unchanged.
• Server-side credentials. The proxy adds Basic auth outbound; Fabric never sees or sends backend secrets.
• Response URL rewriting so HATEOAS-style links stay self-consistent and callable.
• Adding a backend = one app setting + one line of code.

Why it fits Fabric

• MPE is Fabric's native way to reach a private Azure resource; once approved, Fabric resolves the Function App's hostname to a private IP on Microsoft's backbone.
• Flex Consumption gives you VNet integration without an App Service Environment and scales per second.
• No backend changes — the backends keep speaking plain HTTP/Basic-auth on the VNet; they don't know the caller is Fabric.

What's in the repo

End-to-end walkthrough: VNet + subnets, two FastAPI VMs standing in for on-prem servers, the proxy itself, Flex Consumption deployment with VNet integration, storage hardening (the Microsoft.Storage service endpoint gotcha that costs everyone half a day), MPE creation and approval, the Fabric notebook, and a CI/CD path via Azure Repos + Deployment Center.

Repo: DimKal-Org/fabric-mpe-function-proxy - key files: README.md, src/function_app.py, notebooks/fabric_call_function.ipynb.

 

[Gautam_Kumar01]

Great reference architecture @dimkalamaras 

The Managed Private Endpoint + Function App as a reverse proxy pattern is exactly what we needed for connecting Fabric Notebooks to on-prem APIs without exposing public IPs.

Loved the "one catch-all route" and server-side auth approach. Keeps Fabric from ever seeing backend secrets.

Going to try this with our MES system. Quick question: Did you face any cold-start issues with Flex Consumption when hitting multiple backends in parallel?

Thanks for sharing the repo!

[dimkalamaras]

Hi @Gautam_Kumar01 ,

Thanks for your input. The solution described above was implemented and the feature regarding the "always on" instances was enabled (Azure Functions Flex Consumption plan hosting | Microsoft Learn).
So, no we did not witness any cold start issues. 

[Gautam_Kumar01]

Thanks @dimkalamaras for the clarification! 

Good to know about the "always on" instances in Flex Consumption. That solves the cold-start concern for our MES system.

Will implement this MPE + Function App proxy pattern and share results. Appreciate you open-sourcing the repo!