Forum Discussion

wojciech's avatar
wojciech
Helper II
1 year ago
Solved

Data Flow Gen2 Best Practices

Hi all, There are many great resources on Microsoft Learn regarding best practices for Dataflows Gen1: https://learn.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-best-practices ...
Need Help
  • V-yubandi-msft's avatar
    V-yubandi-msft
    1 year ago

    Hi wojciech ,

    Thanks for your response. 

    No, they don’t have to be combined into a single dataflow. You can still maintain separate staging and transformation layers, but the transformation dataflow must explicitly connect to the staging storage (Lakehouse/Warehouse).

     

    Regards,

    Yugandhar.

wojciech's avatar
wojciech
Helper II
1 year ago

Thank you V-yubandi-msft ,

 

Would it mean that the downstream "transformation" dataflow should connect to a staging lakehouse/warehouse or staging and transformation queries must now be a part of a single dataflow?

V-yubandi-msft's avatar
V-yubandi-msft
Community Support
1 year ago

Hi wojciech ,

Thanks for your response. 

No, they don’t have to be combined into a single dataflow. You can still maintain separate staging and transformation layers, but the transformation dataflow must explicitly connect to the staging storage (Lakehouse/Warehouse).

 

Regards,

Yugandhar.

  • wojciech's avatar
    wojciech
    Helper II
    1 year ago

    Thanks, V-yubandi-msft ,

    I did some testing yesterday, and it looks like with staging enabled - since it provisions a Lakehouse/Warehouse behind the scenes - you can implement incremental refresh end-to-end. First, you set the policies on the staging dataflow, and then set them again on the transformation dataflow.

    Appreciate your response!