Forum Discussion

amcpherson's avatar
amcpherson
Helper I
2 years ago

Fabric Firewall Issue

We have been working through implementing fabric lakehouse but keep running into the same firewall issue. Our Azure firewall requires a machine from our network to access the data and this seems to require us to use an on-prem data connection which limits us to dataflows gen 2. Does anyone know a way around this? The amount of data that is being processed takes too long on refresh and we are also forced to load the data to a table instead of files.

 

Thanks!

General Comment
Need Help
Pipelines

5 Replies

  • v-cboorla-msft's avatar
    v-cboorla-msft
    Microsoft Employee
    2 years ago

    Hi amcpherson 

     

    Apologies for the issue you have been facing.

     

    When using Microsoft Fabric Dataflow Gen2 with an on-premises data gateway, you might encounter issues with the dataflow refresh process. The underlying problem occurs when the gateway is unable to connect to the dataflow staging Lakehouse in order to read the data before copying it to the desired data destination. This issue can occur regardless of the type of data destination being used.

    For more details, refer : Set new firewall rules on server running the gateway.

     

    I hope this information is helpful. Please do let us know if you have any questions.

    • amcpherson's avatar
      amcpherson
      Helper I
      2 years ago

      Thanks for the response. We would like to avoid using Dataflows if possible though as we want to load the the data to files as a starting point for the raw data. Do you know a workaround? We are currently evaluating possibly using Azure Data Factory for the time being until the pipeline update is released in Q1 of 2024.

      • v-cboorla-msft's avatar
        v-cboorla-msft
        Microsoft Employee
        2 years ago

        Hi amcpherson 

         

        Apologies for the delay in reply from our side. 

        Currently there is no workaround for this.
        Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

        Hope this helps. Please let me know if you have any further queries.