Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Don't miss out! 2025 Microsoft Fabric Community Conference, March 31 - April 2, Las Vegas, Nevada. Use code MSCUST for a $150 discount. Prices go up February 11th. Register now.

Reply
rdnguyen
Helper V
Helper V

Dataflow chains

I set up a series of dataflows and set refresh for the 3 upstream.

 

This set of dataflows siphon pretty much a couple millions records and I have only 1 hour time to make its data fully flow downstream. I read some that if A > C and B > C, whereas B is lesser stream, its linked entity could be disable in C so that when B is refresh, C would not refresh in linked yet follow A schedule.

 

But when I tried that, my C dataflow got error message as following:

 

Error: On-Prem execution not supported for entity '2d20a598-b3b3-4ffe-9882-6e8db6ba1c99/3e9ec4a0-55ed-45b7-b2e0-64d3e98914e2/ItemProdCls_Active' since it refers to another entity.. RootActivityId = d172af1b-bb8f-4168-b384-be0469fce80f.Param1 = 2d20a598-b3b3-4ffe-9882-6e8db6ba1c99/3e9ec4a0-55ed-45b7-b2e0-64d3e98914e2/ItemProdCls_Active Request ID: d9e73938-69ad-ad3f-b44c-c3e15a65ab60.

 

When B linked entity is enable load again, things get back to normal.

 

Since refresh schedule is not always on time, even though I set B refresh 30 minute later than A, sometimes I noticed both A and B in refresh progress at the same time. My question for this is how will C is processed?

 

Would you have any other suggestion rather than stagering time of refresh schedule? 

 

Thanks for your advise!

 

Best regards.

1 ACCEPTED SOLUTION
v-cazheng-msft
Community Support
Community Support

Hi @rdnguyen ,

 

Is dataflow C built on dataflow A and dataflow B? If you would like to know the refresh logic of linked entities, you may refer to Link entities between dataflows - Power Query | Microsoft Docs. For links between workspaces and Links in the same workspace, the refresh logics are different.

  • Refresh for links from entities in different workspaces behaves like a link to an external data source. When the dataflow is refreshed, it takes the latest data for the entity from the source dataflow. If the source dataflow is refreshed, it doesn't automatically affect the data in the destination dataflow.
  • When data refresh occurs for a source dataflow, that event automatically triggers a refresh process for dependent entities in all destination dataflows in the same workspace, including any calculated entities based on them. All other entities in the destination dataflow are refreshed according to the dataflow schedule. Entities that depend on more than one source refresh their data whenever any of their sources are refreshed successfully.
  • The entire refresh process is committed at once. Because of this, if the data refresh for the destination dataflow fails, the data refresh for the source dataflow fails as well.

 

Best Regards,

Community Support Team _ Caiyun

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly. If I misunderstand your needs or you still have problems on it, please feel free to let me know. Thanks a lot!

View solution in original post

2 REPLIES 2
v-cazheng-msft
Community Support
Community Support

Hi @rdnguyen ,

 

Is dataflow C built on dataflow A and dataflow B? If you would like to know the refresh logic of linked entities, you may refer to Link entities between dataflows - Power Query | Microsoft Docs. For links between workspaces and Links in the same workspace, the refresh logics are different.

  • Refresh for links from entities in different workspaces behaves like a link to an external data source. When the dataflow is refreshed, it takes the latest data for the entity from the source dataflow. If the source dataflow is refreshed, it doesn't automatically affect the data in the destination dataflow.
  • When data refresh occurs for a source dataflow, that event automatically triggers a refresh process for dependent entities in all destination dataflows in the same workspace, including any calculated entities based on them. All other entities in the destination dataflow are refreshed according to the dataflow schedule. Entities that depend on more than one source refresh their data whenever any of their sources are refreshed successfully.
  • The entire refresh process is committed at once. Because of this, if the data refresh for the destination dataflow fails, the data refresh for the source dataflow fails as well.

 

Best Regards,

Community Support Team _ Caiyun

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly. If I misunderstand your needs or you still have problems on it, please feel free to let me know. Thanks a lot!

collinq
Super User
Super User

Hi @rdnguyen ,

 

I have run into this same problem in the past.  There is no "timer" function native to Power BI that would let you daisy-chain the dataflows.  Therefore, the way you are doing it is probably the best way.  It just takes experimentation to get the flow and timing right.  

I have created an idea so please vote for this (and have yoru friends and family vote too) and maybe this will get on Microsoft's radar.

https://ideas.powerbi.com/ideas/idea/?ideaid=e86653d6-adab-ec11-826d-501ac5248c99

 

 




Did I answer your question? Mark my post as a solution!

Proud to be a Datanaut!
Private message me for consulting or training needs.




Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Prices go up Feb. 11th.

Feb2025 Sticker Challenge

Join our Community Sticker Challenge 2025

If you love stickers, then you will definitely want to check out our Community Sticker Challenge!

Jan25PBI_Carousel

Power BI Monthly Update - January 2025

Check out the January 2025 Power BI update to learn about new features in Reporting, Modeling, and Data Connectivity.

Jan NL Carousel

Fabric Community Update - January 2025

Find out what's new and trending in the Fabric community.