Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago
Solved

Execute Data Pipeline Via API

I have been attempting to execute a fabric data factory data pipeline via rest API from a synapse pipeline with no success. Has anyone done this? Is this possible currently? I thought I could do this via the Job Scheduler - Run On Demand Item Job but it does not seem to be working or I am just doing something wrong. There does not seem to be much documentation on the job types but based on the example I should be able to run it with the paramater "DefaultJob" as job type, however, I receive errorCode: Invalid Job Type, message: The requested job type is invalid.

Is this something anyone else has accomplished already?

  • Anonymous's avatar
    Anonymous
    2 years ago

    Hello Anonymous ,

    The Job Scheduler API is missing documentation when it comes to the jobType parameter. If you are trying to run a Data Pipeline on demand, don't use DefaultJob like this.

     

    /jobs/instances?jobType=DefaultJob

     

    Instead, use Pipeline like this.

     

    /jobs/instances?jobType=Pipeline

15 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hello Anonymous ,

    The Job Scheduler API is missing documentation when it comes to the jobType parameter. If you are trying to run a Data Pipeline on demand, don't use DefaultJob like this.

     

    /jobs/instances?jobType=DefaultJob

     

    Instead, use Pipeline like this.

     

    /jobs/instances?jobType=Pipeline

  • mpoeckl's avatar
    mpoeckl
    Microsoft Employee

    One additional comment to the solution:

    1. If you need to pass parameters to the pipeline, you need to provide a payload/body in the following format:

    {   "executionData": {
            "parameters": {
                "YourParameter1": 1,
                "YourParameter2": "Param2
            }
        }
    }

     

    Tip: if you want to call Fabric APIs from a Fabrik Notebook you can use the semantic link library. This is also a great way of verifying/testing the API calls directly within Fabric:

    # Install sempy version 0.4.0 or higher
    !pip install semantic-link --q 
    
    import sempy.fabric as fabric
    
    # Init the sempy client
    client = fabric.FabricRestClient()
    
    # Fabric REST API call “Job Scheduler – Run On Demand Item Job”
    #    Docs: Job Scheduler - Run On Demand Item Job - REST API (Core) | https://learn.microsoft.com/en-us/rest/api/fabric/core/job-scheduler/run-on-demand-item-job
    #    HTTP: POST https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/items/{itemId}/jobs/instances?jobType={jobType}
    #    Body/Payload: {"executionData": { ... } }
    
    # Get the default workspace id of the notebook
    workspaceId = fabric.get_workspace_id()
    itemId = f"yourPipelineUuid"
    jobType = f"Pipeline"
    payload = {
                "executionData": {
                    "parameters": {
                        "YourParamter1": 1,
                        "YourParamter2": "Param2"
                        }
                    }
                }
    
    # Call Fabric REST API with semantic link Fabric REST client (sempy)
    # Run without payload / no or default pipeline parameters
    
    response = client.post(f"/v1/workspaces/{workspaceId}/items/{itemId}/jobs/instances?jobType={jobType}")
    
    # Run with payload / pipeline parameters
    response = client.post(f"/v1/workspaces/{workspaceId}/items/{itemId}/jobs/instances?jobType={jobType}",json= payload)

     

  • Unfortunately fabric Data pipeline APIs have yet not been exposed owining to which it is currently not possible to trigger the fabric data pipeline any way other than manual process or scheduled trigger

  • yllsuarez76's avatar
    yllsuarez76
    Microsoft Employee

    Hello,

    I am getting an error either running a notebook or a pipeline from synapse,