Forum Discussion
Accessing On-Prem API via Python/PySpark in Notebooks
Hi everyone,
We have an on-prem Aruba API endpoint that's accessible through the Power BI gateway.
I'm looking to access this data using Python or PySpark within Microsoft Fabric notebooks.
Has anyone successfully done this or can share guidance on how to set up the connection?
Thanks in advance!
Yevgeny
All right.
This is actually very often the case.However, we can think of two additional ideas.
Mirroring would be another idea if you already have the data in the SQL server or, as you have already written, incremental copy with CDC to a fabric SQL database.
Both are currently still in preview, but according to my tests they work very well.
Unless it is critical, these would be alternatives.Best regards
11 Replies
Hello YevgenyM,
Thank you for reaching out to the Microsoft fabric community forum.Microsoft Fabric notebooks (Python/PySpark) don’t currently support direct connections to on-premises APIs via the Power BI gateway. A recommended workaround is to use an on-premises script to call the Aruba API and push the data to OneLake or Azure Data Lake Storage, which can then be accessed in Fabric notebooks. Configure the gateway to ensure network access to the API, store credentials securely and use PySpark to process the data in Fabric.
Best regards,
Ganesh Singamshetty.Do you know if it is planned in the future to be supported?
Hello YevgenyM,
Currently, direct support for on-premises APIs through the gateway in Fabric notebooks is not available, and Microsoft has not announced a roadmap or ETA for this specific capability.
I’d recommend keeping an eye on the official Microsoft Fabric release notes: https://learn.microsoft.com/en-us/fabric/fundamentals/whats-new and also submitting this idea in the Fabric Ideas forum: https://community.fabric.microsoft.com/t5/Fabric-Ideas/idb-p/fbc_ideas. The more community votes an idea receives, the more likely it is to be considered by the product team for future updates.
Best regards,
Ganesh Singamshetty
Hi YevgenyM,
another option, if available, would be to use an Azure Function that retrieves the data and then stores it.
Of course, this requires a connection to on-premises.Best regards
Nope, we don't have full control over azure services in the organization and prefer to stick to fabric services we do have full control over them.
Currently, it is most likely we write the data to on-prem MSSQL
The next step will be incremental copyjob to bronze lakehouse.
All right.
This is actually very often the case.However, we can think of two additional ideas.
Mirroring would be another idea if you already have the data in the SQL server or, as you have already written, incremental copy with CDC to a fabric SQL database.
Both are currently still in preview, but according to my tests they work very well.
Unless it is critical, these would be alternatives.Best regards
You can't directly use the Power BI gateway from Fabric notebooks. Instead:
- Option A: Pull data into a Power BI dataset via the gateway, then query that dataset from Python/PySpark.
- Option B: Set up a secure cloud-accessible API (via Azure Function or reverse proxy) that talks to the on-prem Aruba API.
- Option C: Use Power Automate or a script to push Aruba API data into OneLake or a Lakehouse, then read it from notebooks.Hello YevgenyM,
Could you please confirm if your query has been resolved by the provided solutions? This would be helpful for other members who may encounter similar issues.
Thank you for being part of the Microsoft Fabric Community.
