Forum Discussion
Dataflow inside Dataflow
Greetings!
I'm creating a dataflow in my workspace, and I realized that it's possible to use another dataflow as source.
What is the difference between the 3 options?
Is it wise to do this? What are the positives and negatives of using data flow in this way?
Are there any best practices for doing this?
Best regards,
Anonymous Microsoft Fabric Dataflows: Data integration and transformation within the Microsoft Fabric environment.
Power Platform Dataflows: Data preparation and transformation across Power Platform applications like Power Apps and Power Automate.
Power BI Dataflows: Data preparation and transformation specifically for Power BI reports and dashboards.Hi Anonymous,
Thanks for reaching out to the Microsoft Fabric Community Forum.Also, great insights from bhanu_gautam appreciate your contribution.
To add more details:
- Microsoft Fabric Dataflows are used within Fabric for big data scenarios, integrating with Lakehouse and Data Warehouse.
- Power Platform Dataflows work across Power BI, Power Apps, and Power Automate, storing data in Azure Data Lake Gen2.
- Power BI Dataflows are an older version of Power BI dataflows and are not recommended for new implementations.
Is it a good practice to use one dataflow inside another?
Yes, but with caution. This is called dataflow chaining, and while it has benefits, it also comes with some risks.
Pros:
- You can standardize data transformations across multiple reports.
- Breaking large datasets into multiple stages can improve processing efficiency.
- Ensures all reports use the same cleaned and transformed data.
Cons:
- Each additional dataflow adds another processing step.
- If the first dataflow fails or changes, it can break downstream dataflows.
- Intermediate results are stored, which may consume more OneLake or Azure Data Lake storage.
Best practices for chaining dataflows:
- Limit chaining to 1-2 levels to avoid long refresh times.
- Use incremental refresh to optimize performance.
- Monitor dependencies to prevent failures when source dataflows change.
- Optimize transformations early to reduce redundant processing in later stages.
If your use case involves big data processing and advanced transformations, Fabric Dataflows are the best option.
If this information is helpful, please “Accept it as a solution” and give a "kudos" to assist other community members in resolving similar issues more efficiently.
Thank you.
7 Replies
v-ssriganeshCommunity Support
Hi Anonymous,
Thanks for reaching out to the Microsoft Fabric Community Forum.Also, great insights from bhanu_gautam appreciate your contribution.
To add more details:
- Microsoft Fabric Dataflows are used within Fabric for big data scenarios, integrating with Lakehouse and Data Warehouse.
- Power Platform Dataflows work across Power BI, Power Apps, and Power Automate, storing data in Azure Data Lake Gen2.
- Power BI Dataflows are an older version of Power BI dataflows and are not recommended for new implementations.
Is it a good practice to use one dataflow inside another?
Yes, but with caution. This is called dataflow chaining, and while it has benefits, it also comes with some risks.
Pros:
- You can standardize data transformations across multiple reports.
- Breaking large datasets into multiple stages can improve processing efficiency.
- Ensures all reports use the same cleaned and transformed data.
Cons:
- Each additional dataflow adds another processing step.
- If the first dataflow fails or changes, it can break downstream dataflows.
- Intermediate results are stored, which may consume more OneLake or Azure Data Lake storage.
Best practices for chaining dataflows:
- Limit chaining to 1-2 levels to avoid long refresh times.
- Use incremental refresh to optimize performance.
- Monitor dependencies to prevent failures when source dataflows change.
- Optimize transformations early to reduce redundant processing in later stages.
If your use case involves big data processing and advanced transformations, Fabric Dataflows are the best option.
If this information is helpful, please “Accept it as a solution” and give a "kudos" to assist other community members in resolving similar issues more efficiently.
Thank you.- v-ssriganesh
Hi Anonymous
,
I hope this information is helpful. Please let me know if you have any further questions or if you'd like to discuss this further. If this answers your question, please accept it as a solution and give it a 'Kudos' so other community members with similar problems can find a solution faster.
Thank you. - meechielvp60
Helper I
I Accepted
- v-ssriganesh
Hi Anonymous,
May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.
Thank you.
- v-ssriganesh
Hi Anonymous,
I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.
Thank you. - bhanu_gautam
Super User
Anonymous Microsoft Fabric Dataflows: Data integration and transformation within the Microsoft Fabric environment.
Power Platform Dataflows: Data preparation and transformation across Power Platform applications like Power Apps and Power Automate.
Power BI Dataflows: Data preparation and transformation specifically for Power BI reports and dashboards. - meechielvp60
You can see every pipeline, connector ect. in the app
