Is it possible to configre vnet data gateway access for Airflow DAGs hosted in Fabric?

[Strebor]

We are trying to migrate our Airflow DAGs from a Kubernetes deployment that has vnet access for its DAGs, to Fabric's Airflow implementation.

We have the vnet added to the Fabric setup already and are able to directly access our VM-hosted SQL Servers in standard copy and pipeline tasks. When setting up connections to the SQL server, we have to make sure to select the vnet as data data gateway dropdown for this to work.

We want to do the same in Airflow - but this doesn't seem to be possible - i.e. the DAGs inside the Airflowjob do not seem to be routed via the Data Gateway and we cannot see any configuration option where we can select the vnet in the same way as we can for other Fabric actions.

Is this a known limitation for the Apache Airflow hosting in Fabric?

[n0elleli]

Hi @Strebor, 

Would love to learn more about your Airflow requirements. Please shoot me an email at noelleli@microsft.com. 

[v-prasare]

We are following up once again regarding your query. Could you please confirm ifyou have raised ideas with Microsoft?
If not please raise idea and upvote it. If we don’t hear back, we’ll go ahead and close this thread.
Should you need further assistance in the future, we encourage you to reach out via the Microsoft Fabric Community Forum and create a new thread. We’ll be happy to help.

Thank you for your understanding and participation.

[Strebor]

Hi - not raised with Microsoft - where do I raise this please?

Seems to be quite core piece of functionality missing from the Fabric managed Airflow implementation - surprised noone else has mentioned it. Seems like it is a regression as it was supported on ADF but now since ADF no longer supports Airflow, we have not option in Fabric.

[v-prasare]

Hi @Strebor ,

Thanks for reaching MS Fabric community support

please refer to below and let me know if this helps?

https://learn.microsoft.com/en-gb/answers/questions/2238062/azure-fabric-airflow-feature-in-vnet-int...

I’d encourage you to submit your detailed feedback and ideas via Microsoft's official feedback channels, such as the Microsoft Fabric Ideas.

Feedback submitted here is often reviewed by the product teams and can lead to meaningful improvement.

Thanks,

prashanth

MS Fabric community support

[Strebor]

Thanks: so the reply to this question shows me that this is NOT possible sadly.

As of now, Airflow jobs in Microsoft Fabric do not support VNet integration similar to Azure Data Factory’s Managed Airflow. There is currently no public documentation or feature availability confirming support for Private Endpoints, VNet peering, or Managed VNet integration within Microsoft Fabric's Airflow offering.

If secure communication with Azure VMs or other resources inside your private VNet is a requirement, you may consider:

• Using ADF Managed Airflow, which supports Managed VNet and private connectivity.
• Deploying Airflow on AKS, which gives full control over networking and access to resources in your VNet.
  
  

Reasons neither of these options work for us:

1. It is not possible for us to set up Managed airflow in ADF any more - it produces an error telling us to use Fabric. See here for announcement 
   
   
2. As stated in original post - we already use AKS but are trying to migrate away from it as it's very devops heavy (requiring management/setup skills after following this tutorial) - we just want a managed version that "just works" (aka Managed) - so were hoping Fabric would give us this. Seems not.