Forum Discussion

angela_n's avatar
angela_n
Frequent Visitor
9 months ago
Solved

Unable to create airflow job

Hello Community,   Since yesterday, I have been trying to create an Airflow job, and I keep getting this error:   {"message"."Request failed with 500(Internal Server Error): {\"error\": {\"code\...
data engineering
  • AntoineW's avatar
    AntoineW
    9 months ago

    Hello angela_n,

     

    Since Apache Airflow in Fabric is a managed service that requires specific backend provisioning, a 500 error usually implies the service failed to allocate the necessary compute resources or containers for your instance.

     

    This is often caused by one of three things:

    • Region unavailability: New feature, so Apache Airflow Jobs are not yet available in all Azure regions where Fabric is deployed.

    • Tenant Setting Restrictions: The feature is disabled at the Tenant level, in preview.

    • Verify Capacity Assignment : Airflow Jobs require a genuine Fabric Capacity (F-SKU) or a Trial capacity. Not possible on Power BI Pro or PPU.

     

    References : 

    - https://learn.microsoft.com/en-us/fabric/data-factory/apache-airflow-jobs-concepts

    - https://learn.microsoft.com/en-us/fabric/data-factory/create-apache-airflow-jobs

     

    Hope it can help you!

    Best regards,

    Antoine

nielsvdc's avatar
nielsvdc
Icon for Super User rankSuper User
9 months ago

Hi angela_n, first check if you capacity region is with the Airflow region availability. If your region is not in the list, that might be the cause of the error.

 

And as mentioned in this similar post, it might be just a temporary glitch and it might be suddenly working tomorrow.

 

Hope this helps. If so, please give a Kudos 👍 or mark as Accepted Solution ✔️.

angela_n's avatar
angela_n
Frequent Visitor
9 months ago

Hi nielsvdc 

 

My capacity is East US, which is listed as supported.

I also tried re-logging in, using a different browser, and clearing the cache, but I'm still getting the same message. At this point, it seems like a platform issue on Microsoft's side.

If this is indeed a known glitch, I'm wondering why it's not reflected on the Service Health page, since everything currently shows as green.

Given that the UI isn't working, I also attempted to create the Airflow job via Terraform. After waiting almost 10 minutes, the operation failed with the error "Context deadline exceeded."

Let me know if there's any update or workaround I may have missed.

Thanks