topic Re: Execute Data Pipeline Via API in Pipelines

Execute Data Pipeline Via API

Anonymous — Mon, 04 Mar 2024 15:30:11 GMT

I have been attempting to execute a fabric data factory data pipeline via rest API from a synapse pipeline with no success. Has anyone done this? Is this possible currently? I thought I could do this via the Job Scheduler - Run On Demand Item Job but it does not seem to be working or I am just doing something wrong. There does not seem to be much documentation on the job types but based on the example I should be able to run it with the paramater "DefaultJob" as job type, however, I receive errorCode: Invalid Job Type, message: The requested job type is invalid.

Is this something anyone else has accomplished already?

Re: Execute Data Pipeline Via API

NandanHegde — Mon, 04 Mar 2024 18:22:04 GMT

Unfortunately fabric Data pipeline APIs have yet not been exposed owining to which it is currently not possible to trigger the fabric data pipeline any way other than manual process or scheduled trigger

Re: Execute Data Pipeline Via API

Anonymous — Thu, 07 Mar 2024 12:54:39 GMT

Hello Anonymous ,

The Job Scheduler API is missing documentation when it comes to the jobType parameter. If you are trying to run a Data Pipeline on demand, don't use DefaultJob like this.

/jobs/instances?jobType=DefaultJob

Instead, use Pipeline like this.

/jobs/instances?jobType=Pipeline

Re: Execute Data Pipeline Via API

Anonymous — Fri, 08 Mar 2024 11:17:43 GMT

Hello Anonymous ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others .
Otherwise, will respond back with the more details and we will try to help .

Re: Execute Data Pipeline Via API

cw_61591 — Fri, 08 Mar 2024 20:28:34 GMT

Thank you I can confirm that this is the correct answer and works.

Re: Execute Data Pipeline Via API

Anonymous — Mon, 11 Mar 2024 08:52:13 GMT

Hi cw_61591 , Anonymous

Glad to know your query got resolved. Please continue using Fabric Community for your further queries.

Re: Execute Data Pipeline Via API

mpoeckl — Wed, 13 Mar 2024 17:02:47 GMT

One additional comment to the solution:

1. If you need to pass parameters to the pipeline, you need to provide a payload/body in the following format:

{ "executionData": { "parameters": { "YourParameter1": 1, "YourParameter2": "Param2 } } }

Tip: if you want to call Fabric APIs from a Fabrik Notebook you can use the semantic link library. This is also a great way of verifying/testing the API calls directly within Fabric:

# Install sempy version 0.4.0 or higher !pip install semantic-link --q import sempy.fabric as fabric # Init the sempy client client = fabric.FabricRestClient() # Fabric REST API call “Job Scheduler – Run On Demand Item Job” # Docs: Job Scheduler - Run On Demand Item Job - REST API (Core) | https://learn.microsoft.com/en-us/rest/api/fabric/core/job-scheduler/run-on-demand-item-job # HTTP: POST https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/items/{itemId}/jobs/instances?jobType={jobType} # Body/Payload: {"executionData": { ... } } # Get the default workspace id of the notebook workspaceId = fabric.get_workspace_id() itemId = f"yourPipelineUuid" jobType = f"Pipeline" payload = { "executionData": { "parameters": { "YourParamter1": 1, "YourParamter2": "Param2" } } } # Call Fabric REST API with semantic link Fabric REST client (sempy) # Run without payload / no or default pipeline parameters response = client.post(f"/v1/workspaces/{workspaceId}/items/{itemId}/jobs/instances?jobType={jobType}") # Run with payload / pipeline parameters response = client.post(f"/v1/workspaces/{workspaceId}/items/{itemId}/jobs/instances?jobType={jobType}",json= payload)

Re: Execute Data Pipeline Via API

yllsuarez76 — Tue, 23 Apr 2024 20:09:35 GMT

Hello,

I am getting an error either running a notebook or a pipeline from synapse,

Re: Execute Data Pipeline Via API

PanuO — Mon, 01 Jul 2024 18:21:56 GMT

I am trying to run a notebook and receiving same error (invalid job type). I tried to use Pipeline and Notebook, but they didn't work. What is suitable jobtype for Notebooks?

Re: Execute Data Pipeline Via API

Anonymous — Thu, 24 Oct 2024 06:44:42 GMT

Hi, I am trying to trigger a pipeline using the HTTPS "Run on Demand" feature and want to automate it with a button click from a model-driven app. Does anyone have any idea how we can achieve this? I tried using Power Automate (Flow), but I wasn't able to find any suitable connector.

Re: Execute Data Pipeline Via API

richbenmintz — Fri, 22 Nov 2024 22:44:48 GMT

please add this to the documentation, the docs make no mention of how you should construct the parameters or the appropriate jobtypes

Re: Execute Data Pipeline Via API

alexjbush — Thu, 23 Jan 2025 00:22:40 GMT

The jobType is `RunNotebook` for Notebooks. See here: https://www.reddit.com/r/MicrosoftFabric/comments/1hmtr86/comment/m41gjqs/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

Re: Execute Data Pipeline Via API

PanuO — Sun, 23 Feb 2025 11:41:08 GMT

Microsoft has published Fabric data pipeline public REST API (Preview) - Microsoft Fabric | Microsoft Learn guide, but I cannot get that Run on-demand item job thing working. It just returns 403.

Re: Execute Data Pipeline Via API

7effrey — Thu, 06 Mar 2025 15:47:16 GMT

i found this:

Fabric data pipeline public REST API (Preview) - Microsoft Fabric | Microsoft Learn

Re: Execute Data Pipeline Via API

Koen_Verbeeck — Mon, 28 Apr 2025 12:47:22 GMT

This should now be possible:
https://sqlkover.com/execute-fabric-data-pipeline-from-azure-data-factory/

Re: Execute Data Pipeline Via API

TOPsDEV — Wed, 21 May 2025 18:20:43 GMT

I've got this to work. Just make a custom connector and that one respect the 202 response and will poll for status after 60 seconds.