topic OnPremise Gateway - Refresh of Dataflow Gen2 fails when writing to a Lakehouse in Dataflow

OnPremise Gateway - Refresh of Dataflow Gen2 fails when writing to a Lakehouse

Profab_07 — Mon, 08 Jul 2024 12:14:20 GMT

We have created an OnPrem Gateway and added it to Microsoft Fabric. Now we would like to load a CSV file from our network drive to a Lakehouse using a Dataflow Gen2.

We know that we need to open the following ports in our firewall for the gateway:

Protocol: TCP Endpoints: *.datawarehouse.pbidedicated.windows.net, *.datawarehouse.fabric.microsoft.com, *.dfs.fabric.microsoft.com Port: 1433

Unfortunately, we are not able to open the wildcard domains for port 1433. We tried to open the URL for the SQL analytics endpoint. So we are able to load the file in a dataflow, but there is a problem when we try to write it to a lakehouse. The dataflow can be published, but the refresh fails:

There was a problem refreshing the dataflow. Please try again later. (Request ID: 0fedcd1e-bc70-44b8-9736-19657d758bcb).

Do you have any idea which firewall configuration is necessary (to make it work without wildcards in a domain)?

Re: OnPremise Gateway - Refresh of Dataflow Gen2 fails when writing to a Lakehouse

Anonymous — Tue, 09 Jul 2024 08:25:43 GMT

Hi Profab_07 ,

When configuring a firewall without using a wildcard domain, try the following steps:

Identify the specific service endpoints and corresponding ports that your Dataflow Gen2 needs to access.
Open the necessary TCP ports: Port 1433 is typically used for SQL database connections, except that other ports may be needed for related services.

You can refer to below document:

Adjust communication settings for the on-premises data gateway | Microsoft Learn

Best Regards,
Adamk Kong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Re: OnPremise Gateway - Refresh of Dataflow Gen2 fails when writing to a Lakehouse

Profab_07 — Tue, 09 Jul 2024 09:07:49 GMT

Hello, after opening port 1433 and port 443 for the domains, it works well. Thank you.

Re: OnPremise Gateway - Refresh of Dataflow Gen2 fails when writing to a Lakehouse

Anonymous — Tue, 09 Jul 2024 09:14:34 GMT

Hi Profab_07 ,

Thank you very much for your feedback.😊

Best Regards,
Adamk Kong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.