Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Don't miss out! 2025 Microsoft Fabric Community Conference, March 31 - April 2, Las Vegas, Nevada. Use code MSCUST for a $150 discount. Prices go up February 11th. Register now.

Reply
Dhruv_Ranpura
Frequent Visitor

Getting On-Prem data from SQL Server to Microsoft Fabric using Dataflow (gen2)

Hi,
I am trying to get the On-prem data from  SQL Server to fabric using data factory-> Dataflows(gen2). Below are the steps which I am following:
1. Hvae installed Gateway with latest version which is working fine.
2. Created one lakehouse.

3. Creating Dataflows(gen2) ->selected Get Data->SQL server database -> connecting with server and database using credentials->selected one table-> chose lakehouse in data destination -> go with Lakehouse(none) option.(attached in snapshot)

Dhruv_Ranpura_1-1710141444045.jpeg

 

4.In choose destination target pane I am selecting the lakehouse which I have created earlier in step 2. 

And when I click on next I am getting error : Something went wrong while retrieving the list of tables.(atached in snapshot).

Dhruv_Ranpura_2-1710141474360.jpeg

 


So why I am getting this error?  It would be great if you guide.

 

1 ACCEPTED SOLUTION

@Dhruv_Ranpura so guys, ok so here's the deal (I am learning a lot about all this firewall stuff, hence why all the back and forth).  

 

port 1434 is irrelevant. 

 

One more thing for future readers:  The endpoint listed on the MS doc page (*.datawarehouse.pbidedicated.windows.net*.datawarehouse.fabric.microsoft.com,  *.dfs.fabric.microsoft.com) are wildcard FQDNs.  So it depends on the firewall what you can or cannot setup from what I've been told.  

 

In our case, we specified the 0.0.0.0/TCP/1433 for outgoing traffic and that did the trick.

 

Oh man, what confusion... finally we got to an understanding. 😥  The Microsoft documentation should seriously be rewritten on this point, because the people who are going to read it are Fabric users, not network admins.

View solution in original post

21 REPLIES 21

OK so the DF is able to talk to the DB successfully via the GW, otherwise there wouldn't be any data coming through to the first DF.  So it's when it goes back to the GW and to the data destination that something funky happens with the TLS protocol.  In other words, 

 

DF2--->GW--->LH

 

which is weird because it doesn't do that for me, and I use the exact same architecture, which in my case got rid of all these weird errors:

 

on-prem DB--->GW--->DF1 ---> DF2--->GW--->LH

 

Also, from what I have seen, there is no real tweaking available at the GW UI for TLS.  Only the firewall rules.  Speaking of which, my sysadmin couldn't get this endpoint to respond: *.dfs.fabric.microsoft.com, as if it were misconfigured.  But ETL still works, so not sure how essential it is to have this endpoint in the firewall rules.

 

I sure wish Microsoft would provide some in detail and specific guidance on this, if only with a detailed and cleared doc write-up, ie more than just what can be found here:  On-premises data gateway considerations for data destinations in Dataflow Gen2 - Microsoft Fabric | ...  I mean, we are in the weeds here, and if this product is meant for BI or data people but NOT network engineers, well then... we got a problem now, don't we?

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

Jan NL Carousel

Fabric Community Update - January 2025

Find out what's new and trending in the Fabric community.

Top Solution Authors