Forum Discussion
Gen 2 Dataflow unable to use other dataflow source
A dataflow we have created is unable to get data from another dataflow.
Steps to reproduce:
- Create a Gen 2 dataflow. The datasource is an Azure SQL database and it the dataflow is connection through a VNet data gateway.
- Create a second Gen 2 dataflow.
- Get data from a datalfow in the second dataflow.
- Select the first Gen 2 dataflow.
We are not able to see any of the first dataflow's queries. We get the error:
"Microsoft SQL: A connection was successfully established with the server, but then an error occurred during the login process. An existing connection was forcibly closed by the remote host"
In the past we have successfully had a dataflow use another dataflow as a datasource. However, the difference this time is that in this case, the first dataflow is getting data from an Azure SQL database through a VNet Data Gateway.
Is there any way this can be fixed so the seconc dataflow can get data from the first dataflow?
Hi Lorenz33,
As Murtaza_Ghafoor suggested, what I would recommend is have your first dataflow write to a lakehouse/warehouse, and then have your second dataflow pull from that lakehouse/warehouse.
Dataflow Gen 2 works differently than dataflow Gen 1. Gen 1 acted as a data store for you to use in other queries/reports, however gen 2 is designed for data movement/transformation, not as data storage, so having your first data flow store the data in a lakehouse/warehouse puts it in a place that is easily accessible from the rest of the Fabric platform.
Here are the docs on adding destinations in dataflow gen 2: Dataflow Gen2 data destinations and managed settings - Microsoft Fabric | Microsoft Learn
5 Replies
PROBLEM STATEMENT:
It seems like the host connection is blocking the access because 2nd Data flow you created tries to login into managed SQL connections (via v net gateway) Dataflow 1 established connection.
I am afraid this approach is not applicable.
PROPOSED SOLUTION:
You need to follow this approach:
Save Dataflow 1 output to OneLake (Lakehouse/Warehouse) andRead that data in Dataflow 2 from Lakehouse/Warehouse instead of chaining dataflows.
If this helped, ✓ Mark as Solution | Kudos appreciated |Help OthersHi Lorenz33,
As Murtaza_Ghafoor suggested, what I would recommend is have your first dataflow write to a lakehouse/warehouse, and then have your second dataflow pull from that lakehouse/warehouse.
Dataflow Gen 2 works differently than dataflow Gen 1. Gen 1 acted as a data store for you to use in other queries/reports, however gen 2 is designed for data movement/transformation, not as data storage, so having your first data flow store the data in a lakehouse/warehouse puts it in a place that is easily accessible from the rest of the Fabric platform.
Here are the docs on adding destinations in dataflow gen 2: Dataflow Gen2 data destinations and managed settings - Microsoft Fabric | Microsoft Learn
Hello Lorenz33,
Power BI Gen2 dataflows cannot use another dataflow as a source if the first dataflow connects to its source via a VNet data gateway.
Linked tables work only when both dataflows are in a Premium workspace and the upstream dataflow’s output is accessible without gateway restrictions.
Workarounds:
- Have the second dataflow query the same SQL source directly using its own gateway.
- Persist the first dataflow’s output to ADLS Gen2 or a Power BI dataset, then have the second dataflow read from there.
References:
Linked tables
Dataflow limitationsHi Lorenz33
Thank you for posting your query in the Microsoft Fabric Community Forum, and thanks to Murtaza_Ghafoor , Olufemi7 and tayloramy for sharing valuable insights.
Could you please confirm if your query has been resolved by the provided solution?If you have any more questions, please let us know and we’ll be happy to help.
Regards,
Microsoft Fabric Community Support Team.
Hi Lorenz33
Just checking in as we haven't received a response to our previous message. Were you able to review the information above? Let us know if you have any additional questions.
Thank You.
