Forum Discussion
Power ParticipantDataflow Gen2 Issue
Hi, I'm getting the same error.
My source is a SQL on premise database, my destination is a lakehouse and I'm using a Dataflow Gen2 via a gateway (version 3000.178.9 (June 2023)) to transfer data into the lakehouse. The same gateway is being used by Power BI and Data Factory with no issues, which leads me to believe it wouldn't be a firewall or gateway service account issue, as otherwise the problem with connecting would exist there too?
Can anyone please help or advise? Am I missing something?
TIA.
Community SupportThe underlying issue is that the data gateway cannot connect to the dataflow staging LakeHouse in order to read the data before copying it to the desired output destination. Reading data from a lakehouse uses TCP over port 1433, which is used to copy the data from the staging lakehouse to the output destination. So the firewall rules on the data gateway server and/or customer's proxy servers need to be updated to allow outbound traffic from the data gateway server to the following
Protocol: TCP
Endpoint: *.datawarehouse.pbidedicated.windows.net
Port: 1433
When you use the same data gateway for Power BI and Data Factory with no issues, it is because their traffics flow to different Endpoints or they use different protocols/ports that are not blocked on the data gateway server.
Best Regards,
Community Support Team _ Jing
You saved my day! My firewall was filtering the traffic flow you mention!
- Jeanxyz
I have been having the same issue for weeks and couldn't fix it with IT security. So I understand the problem is staging lakehouse couldn't read data from on-prem gateway, but then you said "Reading data from a lakehouse uses TCP over port 1433", but we need to ensure Fabric can read data from on-prem gateway to lakehouse? deblock TCP over 1433 won't solve the issue, right?
for your info, our IT has deblocked protal 1433, but the problem remains. I have also distabled "Enable staging". It doens't help neither
This response was what we had used when I ha dthe same problem. Gen1 DFs used Port 443 which is relatively saf ebut Gen2 Dfs use Port 1433 for write backs. It is very risky to open that port indefinitely. So your network team has to monitor all requests coming from fabric (there is a list of allowed ports on the Microsoft Fabric Website) and then allow the fabric related IPs / addresses to go through. It may require a couple of trials to add all of them. Mind you the list may also change so check with Fabric support for such changes.
