Forum Discussion
PowerBi Dataset Refresh Using Data Pipelines
Hi All,
I have a big dataset, it has around 15dim tables and a single big fact table and i created quarterly partitions for that fact able using tabuler editor and enabled the sequential refresh using fabric component called data pipelines. Pirpelines are working fine and refresh also fine.
in my pipeline i created nearly 20+ web activites to run each table sequentially.
Web activity is using method type called "POST"
But there is request that running pipeline is a scheduled based but i want to run using rest API after my backend suource level load completed an API should trigger my data pipeline to start.
So need your end to end steps to fullfil this gap.
Please do let me know if any other are details required. Thank you so much for your support in advance.
Thanks,
Lavan
Hi lavankumar1989a , Thank you for reaching out to the Microsoft Community Forum.
Please follow below steps to Trigger the Data Pipeline Using REST API:- Navigate to your data pipeline in the Fabric portal. The URL will contain both the Workspace ID and Pipeline ID, retrieve them.
- Use Azure Active Directory (AAD) to authenticate and obtain a bearer token. For detailed instructions, follow this link:
https://learn.microsoft.com/en-us/rest/api/fabric/articles/get-started/fabric-api-quickstart
- Construct the API Request: HTTP, Method:- POST, Authorization: Bearer {your_access_token}, Content-Type: application/json, for a basic trigger, the body can be empty or contain specific execution parameters if required.
Example endpoint:
https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/items/{pipelineId}/jobs/instances?jobType=Pipeline
- Use your preferred HTTP client or tool to send the POST request.
- This API triggers the entire pipeline, but does not control the order of execution of web activities. Ensure that sequential execution is defined inside the pipeline using dependencies between activities.
- By integrating this API call into the backend process, you can ensure that the data pipeline runs immediately after source-level load completes.
- For more details, please refer below documents:
https://learn.microsoft.com/en-us/fabric/data-factory/pipeline-rest-api
https://learn.microsoft.com/en-us/fabric/data-factory/pipeline-rest-api-capabilities
If this helped solve the issue, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details, always happy to help.
Thank you.
4 Replies
Hi lavankumar1989a , Thank you for reaching out to the Microsoft Community Forum.
Please follow below steps to Trigger the Data Pipeline Using REST API:- Navigate to your data pipeline in the Fabric portal. The URL will contain both the Workspace ID and Pipeline ID, retrieve them.
- Use Azure Active Directory (AAD) to authenticate and obtain a bearer token. For detailed instructions, follow this link:
https://learn.microsoft.com/en-us/rest/api/fabric/articles/get-started/fabric-api-quickstart
- Construct the API Request: HTTP, Method:- POST, Authorization: Bearer {your_access_token}, Content-Type: application/json, for a basic trigger, the body can be empty or contain specific execution parameters if required.
Example endpoint:
https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/items/{pipelineId}/jobs/instances?jobType=Pipeline
- Use your preferred HTTP client or tool to send the POST request.
- This API triggers the entire pipeline, but does not control the order of execution of web activities. Ensure that sequential execution is defined inside the pipeline using dependencies between activities.
- By integrating this API call into the backend process, you can ensure that the data pipeline runs immediately after source-level load completes.
- For more details, please refer below documents:
https://learn.microsoft.com/en-us/fabric/data-factory/pipeline-rest-api
https://learn.microsoft.com/en-us/fabric/data-factory/pipeline-rest-api-capabilities
If this helped solve the issue, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details, always happy to help.
Thank you.Hi lavankumar1989a , Thank you for reaching out to the Microsoft Community Forum.
Please let us know if your issue is solved. If it is, consider marking the answer that helped 'Accept as Solution', so others with similar queries can find it easily. If not, please share the details.
Thank you.Hi @lavankumar1989a , Thank you for reaching out to the Microsoft Community Forum.
Please let us know if your issue is solved. If it is, consider marking the answer that helped 'Accept as Solution', so others with similar queries can find it easily. If not, please share the details.
Thank you.Hi @lavankumar1989a , Thank you for reaching out to the Microsoft Community Forum.
Please let us know if your issue is solved. If it is, consider marking the answer that helped 'Accept as Solution', so others with similar queries can find it easily. If not, please share the details.
Thank you.