topic Re: On-prem gateway firewall rules? in Dataflow

On-prem gateway firewall rules?

ScottPowell — Mon, 12 Jun 2023 02:21:35 GMT

Hi, we're hitting issues trying to load data from an on-prem SQL server database to Fabric. It appears that our on-prem gateway servers need to be able to talk to the SQL endpoints, which they currently can't (blocked by firewall).

Is there clear documentation on exactly what I need to request to our networking team? Not looking forward to this and want to make sure I get this right the first time.

Thanks!

Scott

Re: On-prem gateway firewall rules?

bcdobbs — Mon, 12 Jun 2023 11:52:30 GMT

Hi ScottPowell ,

Had exactly the same issue.

I found most of confirmation in Connectivity to data warehousing - Microsoft Fabric | Microsoft Learn

We've just added *.datawarehouse.pbidedicated.windows.net and allowed communication on 1433.

Having pinged it it also seemed to redirect to *.uksouth.cloudapp.azure.com so I asked IT to add that at the same time to avoid messing them around although don't think that's strictly needed.

It is however now fully working.

Ben

Re: On-prem gateway firewall rules?

suidino2 — Tue, 13 Jun 2023 01:20:09 GMT

The dataflow gen 2 working with On-Prem is too troublesome. I think they need to focus on the context of on-Prem. Other than On-prem, it is nice functioning.

Re: On-prem gateway firewall rules?

ScottPowell — Tue, 13 Jun 2023 01:49:28 GMT

Hi Ben - thank you very much for the info. I didn't see anything that specifically mentioned *.datawarehouse.pbidedicated.windows.net in the link you referenced - do we know that's the "final" URL? Or did you infer that based on the connection string?

In any case, I really appreciate the help, now I'll see how far I can get with the network / firewall team.

Thanks!

Scott

Re: On-prem gateway firewall rules?

ScottPowell — Tue, 13 Jun 2023 01:53:57 GMT

suidino2 I couldn't agree more. I guess it makes sense to start with capabilities that stay in the Microsoft cloud - but then you're only serving customers who honestly already have their stuff in the cloud, and fabric makes potentially less sense.

I have dreams of nightly copying our 1.5 TB data warehouse into Fabric, but so far haven't been able to move a single byte of data from on-prem up. And I've got a gateway cluster with 3 machines in it - but it's going to have to run orders of magnitude faster for this to work.

The one thing that gives me hope is that if I look at the COPY utility documentation, it seems completely doable - and Microsoft has confirmed they are going to have COPY functionality working via on-prem gateway before GA. Can't wait for the day!

Scott

Re: On-prem gateway firewall rules?

bcdobbs — Sat, 17 Jun 2023 08:19:57 GMT

Yes just inferred from connection string but I don't see why that would change between now and GA. Only my guess though!

Re: On-prem gateway firewall rules?

SidJay — Sun, 18 Jun 2023 08:01:50 GMT

In addition to setting a firewall rule, please upgrade to the latest Gateway (version 3000.178.9 released on June 15th). That should help address at least one class of errors involving output destinations and/or queries that depend on other staged queries.

Thanks.