topic Call Fabric data pipelines using notebooks via API's in Data Engineering

Call Fabric data pipelines using notebooks via API's

Anonymous — Fri, 07 Mar 2025 11:54:29 GMT

Hi Community!
Greetings!

I am currently working on an interesting solution for which I need to call pipelines via Rest API, i see there are already many questions posted on this on community regarding the topic. However, none of them talks about the steps or way to generate access token. I would like to know how to generate access token using Azure AD as service principle authentication is not supported for this API call.

Reference: https://learn.microsoft.com/en-us/rest/api/fabric/core/job-scheduler/run-on-demand-item-job

I request your assistance here.

Thanks in advance!

Re: Call Fabric data pipelines using notebooks via API's

spencer_sa — Sat, 08 Mar 2025 14:40:49 GMT

1) Easiest way of calling the Fabric APIs from within a Fabric notebook is to use semantic-link-labs (aka sempy) and create an instance of FabricRESTClient, which uses the credentials of the logged in user to run the API
https://fabric.guru/using-fabricrestclient-to-make-fabric-rest-api-calls

2) If you really want to generate an access token yourself there are 2-3 ways. Firstly you can use the msal python library
https://learn.microsoft.com/en-us/entra/msal/python/

3) Secondly you can craft your token request using the azure identities libraries
https://learn.microsoft.com/en-us/azure/communication-services/quickstarts/identity/access-tokens?tabs=windows&pivots=programming-language-python

4) Lastly you can try to use the requests library (although tbh I've only ever got these working with service principals, which as you note won't kick a pipeline off.

If this helps, please Accept as a Solution to help others find it more easily.

Re: Call Fabric data pipelines using notebooks via API's

Anonymous — Sat, 08 Mar 2025 06:45:57 GMT

Hi @spencer_sa

Much appreciated, since I had to automate this process with scheduling notebook. Fabric Rest Client worked for me. (Apparently idk if it uses the users stored credential as for testing I closed my Fabric window and it still worked.)

The other options however cannot be automated as there will either be a login page(like msal) and import requests definitely didn't work as it is specific to service principals.

Hope this helps for anyone else looking!