Execute Data Pipeline Via API

[cw_615]

I have been attempting to execute a fabric data factory data pipeline via rest API from a synapse pipeline with no success. Has anyone done this? Is this possible currently? I thought I could do this via the Job Scheduler - Run On Demand Item Job but it does not seem to be working or I am just doing something wrong. There does not seem to be much documentation on the job types but based on the example I should be able to run it with the paramater "DefaultJob" as job type, however, I receive errorCode: Invalid Job Type, message: The requested job type is invalid.

Is this something anyone else has accomplished already?

[yllsuarez76]

Hello,

I am getting an error either running a notebook or a pipeline from synapse,

[mpoeckl]

One additional comment to the solution:

1. If you need to pass parameters to the pipeline, you need to provide a payload/body in the following format:

{   "executionData": {
        "parameters": {
            "YourParameter1": 1,
            "YourParameter2": "Param2
        }
    }
}

Tip: if you want to call Fabric APIs from a Fabrik Notebook you can use the semantic link library. This is also a great way of verifying/testing the API calls directly within Fabric:

# Install sempy version 0.4.0 or higher
!pip install semantic-link --q 

import sempy.fabric as fabric

# Init the sempy client
client = fabric.FabricRestClient()

# Fabric REST API call “Job Scheduler – Run On Demand Item Job”
#    Docs: Job Scheduler - Run On Demand Item Job - REST API (Core) | https://learn.microsoft.com/en-us/rest/api/fabric/core/job-scheduler/run-on-demand-item-job
#    HTTP: POST https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/items/{itemId}/jobs/instances?jobType={jobType}
#    Body/Payload: {"executionData": { ... } }

# Get the default workspace id of the notebook
workspaceId = fabric.get_workspace_id()
itemId = f"yourPipelineUuid"
jobType = f"Pipeline"
payload = {
            "executionData": {
                "parameters": {
                    "YourParamter1": 1,
                    "YourParamter2": "Param2"
                    }
                }
            }

# Call Fabric REST API with semantic link Fabric REST client (sempy)
# Run without payload / no or default pipeline parameters

response = client.post(f"/v1/workspaces/{workspaceId}/items/{itemId}/jobs/instances?jobType={jobType}")

# Run with payload / pipeline parameters
response = client.post(f"/v1/workspaces/{workspaceId}/items/{itemId}/jobs/instances?jobType={jobType}",json= payload)

[richbenmintz]

please add this to the documentation, the docs make no mention of how you should construct the parameters or the appropriate jobtypes

[Anonymous · Accepted solution]

Hello @cw_615 ,

The Job Scheduler API is missing documentation when it comes to the jobType parameter. If you are trying to run a Data Pipeline on demand, don't use DefaultJob like this.

/jobs/instances?jobType=DefaultJob

Instead, use Pipeline like this.

/jobs/instances?jobType=Pipeline

[sachinrana205]

Hi, I am trying to trigger a pipeline using the HTTPS "Run on Demand" feature and want to automate it with a button click from a model-driven app. Does anyone have any idea how we can achieve this? I tried using Power Automate (Flow), but I wasn't able to find any suitable connector.

[PanuO]

I am trying to run a notebook and receiving same error (invalid job type). I tried to use Pipeline and Notebook, but they didn't work. What is suitable jobtype for Notebooks?

[alexjbush]

The jobType is `RunNotebook` for Notebooks. See here: https://www.reddit.com/r/MicrosoftFabric/comments/1hmtr86/comment/m41gjqs/?utm_source=share&utm_medi...

[cw_61591]

Thank you I can confirm that this is the correct answer and works.

[Anonymous]

Hi @cw_61591 , @cw_615  

Glad to know your query got resolved. Please continue using Fabric Community for your further queries.

[Anonymous]

Hello @cw_615 ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others .
Otherwise, will respond back with the more details and we will try to help .

[NandanHegde]

Unfortunately fabric Data pipeline APIs have yet not been exposed owining to which it is currently not possible to trigger the fabric data pipeline any way other than manual process or scheduled trigger

[PanuO]

Microsoft has published Fabric data pipeline public REST API (Preview) - Microsoft Fabric | Microsoft Learn guide, but I cannot get that Run on-demand item job thing working. It just returns 403.