Forum Discussion

wojciech's avatar
wojciech
Helper II
1 year ago
Solved

Data Flow Gen2 Best Practices

Hi all, There are many great resources on Microsoft Learn regarding best practices for Dataflows Gen1: https://learn.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-best-practices ...
Need Help
  • V-yubandi-msft's avatar
    V-yubandi-msft
    1 year ago

    Hi wojciech ,

    Thanks for your response. 

    No, they don’t have to be combined into a single dataflow. You can still maintain separate staging and transformation layers, but the transformation dataflow must explicitly connect to the staging storage (Lakehouse/Warehouse).

     

    Regards,

    Yugandhar.

Akash_Varuna's avatar
Akash_Varuna
Super User
1 year ago

Hi wojciech  Dataflow Gen2 removes linked entities but keeps the staging-transformation pattern viable with adjustments. Use Staging Dataflows to export raw data with staging enabled, and connect Transformation Dataflows directly to the storage account tied to the staging output. Ensure correct storage setup and configurations are done properly.

  • wojciech's avatar
    wojciech
    Helper II
    1 year ago

    Hey Akash_Varuna,

    Thanks for coming back to me, much appreciated!

    Could you explain what do you mean by connecting directly to the storage account tied to the staging output. Do you mean connecting to:

    - staging dataflow through dataflow connector

    - staging warehouse/lakehouse directly

    - actual ADLS account for staging lakehouse/warehouse - not sure that's even possible

     

    "Ensure correct storage setup and configurations are done properly" - is there a documantation explaining what proper configuration looks like by any chance?

     

    Many Thanks,

     

    WJ