Fabric Firewall Issue

[amcpherson]

We have been working through implementing fabric lakehouse but keep running into the same firewall issue. Our Azure firewall requires a machine from our network to access the data and this seems to require us to use an on-prem data connection which limits us to dataflows gen 2. Does anyone know a way around this? The amount of data that is being processed takes too long on refresh and we are also forced to load the data to a table instead of files.

Thanks!

[v-cboorla-msft]

Hi @amcpherson 

Apologies for the issue you have been facing.

When using Microsoft Fabric Dataflow Gen2 with an on-premises data gateway, you might encounter issues with the dataflow refresh process. The underlying problem occurs when the gateway is unable to connect to the dataflow staging Lakehouse in order to read the data before copying it to the desired data destination. This issue can occur regardless of the type of data destination being used.

For more details, refer : Set new firewall rules on server running the gateway.

I hope this information is helpful. Please do let us know if you have any questions.

[amcpherson]

Thanks for the response. We would like to avoid using Dataflows if possible though as we want to load the the data to files as a starting point for the raw data. Do you know a workaround? We are currently evaluating possibly using Azure Data Factory for the time being until the pipeline update is released in Q1 of 2024.

[v-cboorla-msft]

Hi @amcpherson 

Apologies for the delay in reply from our side. 

Currently there is no workaround for this.
Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

Hope this helps. Please let me know if you have any further queries.

[v-cboorla-msft]

Hi @amcpherson

We haven’t heard from you on the last response and was just checking back to see if you got a chance to share the feedback.

Please let us know if you have any further questions.

Glad to help.

[amcpherson]

@v-cboorla-msft we are going to use a workaround for now staging the data in a blob.

Thanks,

Alex