Forum Discussion
Questions about functionality about the Apache Airflow Job in Fabric
- 1 year ago
Yes, it is possible to trigger a Fabric Pipeline using Azure Logic Apps:
- Start by creating a new Logic App in the Azure portal. You can choose a trigger that suits your scenario, such as an HTTP request, a schedule, or an event.
- In the Logic App, add an action to call the Fabric Pipeline. This can be done using the HTTP action to make a REST API call to the Fabric Pipeline endpoint.
- Ensure your Fabric Pipeline is set up to accept triggers from external sources. You might need to configure authentication and permissions to allow the Logic App to trigger the pipeline.
Reference: https://learn.microsoft.com/en-us/answers/questions/1805816/accessing-fabric-lakehouse-via-logic-app
When triggering a DAG from the CLI, the REST API or the UI, it is possible to pass configuration for a DAG Run as a JSON blob.
Check these Airflow settings for parallel runs:
Max number of concurrent tasks across the whole Airflow environment,
Max number of concurrent tasks per DAG,
if you are you using a Pool to limit concurrency, check if both tasks are in a pool that has only 1 slot, That can cause serial execution.
What executor are you using?
If you’re on SequentialExecutor, that’s the bottleneck.
Thanks and Regards
Chaithra
Hi b_schmitt ,
We wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
If our response addressed, please mark it as Accept as solution and click Yes if you found it helpful.
Regards,
Chaithra.