Forum Discussion

Lorenz33's avatar
Lorenz33
Helper IV
6 months ago
Solved

Gen 2 Dataflow unable to use other dataflow source

A dataflow we have created is unable to get data from another dataflow.   Steps to reproduce: Create a Gen 2 dataflow. The datasource is an Azure SQL database and it the dataflow is connection th...
Need Help
  • tayloramy's avatar
    tayloramy
    6 months ago

    Hi Lorenz33, 

    As Murtaza_Ghafoor suggested, what I would recommend is have your first dataflow write to a lakehouse/warehouse, and then have your second dataflow pull from that lakehouse/warehouse.

     

    Dataflow Gen 2 works differently than dataflow Gen 1. Gen 1 acted as a data store for you to use in other queries/reports, however gen 2 is designed for data movement/transformation, not as data storage, so having your first data flow store the data in a lakehouse/warehouse puts it in a place that is easily accessible from the rest of the Fabric platform. 

     

    Here are the docs on adding destinations in dataflow gen 2: Dataflow Gen2 data destinations and managed settings - Microsoft Fabric | Microsoft Learn

     

Murtaza_Ghafoor's avatar
Murtaza_Ghafoor
Super User
6 months ago

Lorenz33 

PROBLEM STATEMENT:
It seems like the host connection is blocking the access because 2nd Data flow you created tries to login into managed SQL connections (via v net gateway) Dataflow 1 established connection.
I am afraid this approach is not applicable.

PROPOSED SOLUTION:

You need to follow this approach:

Save Dataflow 1 output to OneLake (Lakehouse/Warehouse) and

Read that data in Dataflow 2 from Lakehouse/Warehouse instead of chaining dataflows.


If this helped, ✓ Mark as Solution | Kudos appreciated |Help Others