Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Score big with last-minute savings on the final tickets to FabCon Vienna. Secure your discount

Get ApacheAirflowJob Starter pool status via API using service principle credentials

I need a way to check the status of a starter pool in an apacheAirflowJob remotely, so that my infrastructure automation may respond. It will also be useful to be able to start the pool if in stopped state. I have a use case where I need to run a short job once per day, but sometimes if the starter pool is paused there is no way to wake it up (even by trying to run the job as the docs suggest). It is as though airflow doesn't exist. Being able to check the status, similar to the getStatus api the page uses would allow a terraform script to respond appropriately (and then request to start the pool thus making it available to the airflow job). This is required stuff for automated business workflows. I need this to run daily at a very unsociable hour, but only once and for the remainder of the day do not need the airflow tools, so starter pool with its power down after 20 minutes of inactivity is perfect for a small workload like this. It's a nice solution using git repo to store dags and a cost efficient pool. However, it is not a viable solution until it can be included in automation scripts. Please add the APIs to poll status and start/stop the pool so that a managed identity or service principle may be used to make the requests and recieve responses if they have Power BI delegated and application permissions and are included in the list of workspace users.

Status: New