topic Re: Using Terraform with Fabric and Airflow in Data Engineering

Using Terraform with Fabric and Airflow

angela_n — Wed, 16 Apr 2025 21:46:03 GMT

Hey everyone,
I’m using Terraform to set up my fabric elements, and I want to use Airflow for orchestration instead of pipelines. I know Terraform is still in the early stages, but I think it’s going to be super beneficial in the long run.
Does anyone know how I can upload the DAG definitions to Fabric?

Thanks,

Re: Using Terraform with Fabric and Airflow

Anonymous — Thu, 17 Apr 2025 09:05:16 GMT

Hi @angela_n ,
Thank you for reaching out to us on Microsoft Fabric Community Forum!

While Terraform is great for provisioning Fabric resources, uploading Airflow DAGs directly via Terraform isn’t supported yet. DAGs need to be placed in the dags/ folder manually or through a CI/CD pipeline after setup. A common approach is to use Terraform for infrastructure and a separate script or pipeline to upload DAGs. This would be a great feature to have, so we recommend submitting it to the Fabric Ideas forum.
Feedback submitted through these channels is frequently reviewed by the product teams and can contribute to meaningful improvements.

Hope this helps.If so,consider accepting it as solution.

Regards,
Pallavi.

Re: Using Terraform with Fabric and Airflow

angela_n — Thu, 17 Apr 2025 14:30:09 GMT

Hi Pallavi,

I get that uploading Airflow DAGs directly via Terraform isn’t supported yet and that they need to be placed in the dags/ folder manually. Could you give me some tips on how to set up a CI/CD pipeline for this, right now is showing as 'Unsupported'?

thanks,

Re: Using Terraform with Fabric and Airflow

angela_n — Thu, 17 Apr 2025 15:25:59 GMT

Hi Pallavi,

I think I get what you mean 😉. I did not see this config before

Thanks for your help.

Re: Using Terraform with Fabric and Airflow

Anonymous — Mon, 21 Apr 2025 08:51:52 GMT

Hi @angela_n ,
I hope the answer was helpful,If the issue has been resolved, please mark the helpful reply as a "solution" to indicate that the question has been answered and to assist others in the community.

Thank you for your cooperation. Have a great day.

Re: Using Terraform with Fabric and Airflow

aterry_sit — Tue, 22 Jul 2025 10:32:40 GMT

Hi, Im also trying to deploy DAGs to Airflow in Fabric, how would you deploy them through CI/CD? It doesnt looks like any Fabric CLI supports uplodaing DAG files at all and the storage account that Fabric uses doesnt seem to be exposed?

Re: Using Terraform with Fabric and Airflow

angela_n — Thu, 24 Jul 2025 17:39:42 GMT

Hi @aterry_sit , we ended up installing our instance of airflow, outside fabric, and we update the DAGs using Azure Devops pipelines. Airflow is not production ready app within fabric