Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
rdnguyen
Helper V
Helper V

Dataflow chains

I set up a series of dataflows and set refresh for the 3 upstream.

 

This set of dataflows siphon pretty much a couple millions records and I have only 1 hour time to make its data fully flow downstream. I read some that if A > C and B > C, whereas B is lesser stream, its linked entity could be disable in C so that when B is refresh, C would not refresh in linked yet follow A schedule.

 

But when I tried that, my C dataflow got error message as following:

 

Error: On-Prem execution not supported for entity '2d20a598-b3b3-4ffe-9882-6e8db6ba1c99/3e9ec4a0-55ed-45b7-b2e0-64d3e98914e2/ItemProdCls_Active' since it refers to another entity.. RootActivityId = d172af1b-bb8f-4168-b384-be0469fce80f.Param1 = 2d20a598-b3b3-4ffe-9882-6e8db6ba1c99/3e9ec4a0-55ed-45b7-b2e0-64d3e98914e2/ItemProdCls_Active Request ID: d9e73938-69ad-ad3f-b44c-c3e15a65ab60.

 

When B linked entity is enable load again, things get back to normal.

 

Since refresh schedule is not always on time, even though I set B refresh 30 minute later than A, sometimes I noticed both A and B in refresh progress at the same time. My question for this is how will C is processed?

 

Would you have any other suggestion rather than stagering time of refresh schedule? 

 

Thanks for your advise!

 

Best regards.

1 ACCEPTED SOLUTION
v-cazheng-msft
Community Support
Community Support

Hi @rdnguyen ,

 

Is dataflow C built on dataflow A and dataflow B? If you would like to know the refresh logic of linked entities, you may refer to Link entities between dataflows - Power Query | Microsoft Docs. For links between workspaces and Links in the same workspace, the refresh logics are different.

  • Refresh for links from entities in different workspaces behaves like a link to an external data source. When the dataflow is refreshed, it takes the latest data for the entity from the source dataflow. If the source dataflow is refreshed, it doesn't automatically affect the data in the destination dataflow.
  • When data refresh occurs for a source dataflow, that event automatically triggers a refresh process for dependent entities in all destination dataflows in the same workspace, including any calculated entities based on them. All other entities in the destination dataflow are refreshed according to the dataflow schedule. Entities that depend on more than one source refresh their data whenever any of their sources are refreshed successfully.
  • The entire refresh process is committed at once. Because of this, if the data refresh for the destination dataflow fails, the data refresh for the source dataflow fails as well.

 

Best Regards,

Community Support Team _ Caiyun

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly. If I misunderstand your needs or you still have problems on it, please feel free to let me know. Thanks a lot!

View solution in original post

2 REPLIES 2
v-cazheng-msft
Community Support
Community Support

Hi @rdnguyen ,

 

Is dataflow C built on dataflow A and dataflow B? If you would like to know the refresh logic of linked entities, you may refer to Link entities between dataflows - Power Query | Microsoft Docs. For links between workspaces and Links in the same workspace, the refresh logics are different.

  • Refresh for links from entities in different workspaces behaves like a link to an external data source. When the dataflow is refreshed, it takes the latest data for the entity from the source dataflow. If the source dataflow is refreshed, it doesn't automatically affect the data in the destination dataflow.
  • When data refresh occurs for a source dataflow, that event automatically triggers a refresh process for dependent entities in all destination dataflows in the same workspace, including any calculated entities based on them. All other entities in the destination dataflow are refreshed according to the dataflow schedule. Entities that depend on more than one source refresh their data whenever any of their sources are refreshed successfully.
  • The entire refresh process is committed at once. Because of this, if the data refresh for the destination dataflow fails, the data refresh for the source dataflow fails as well.

 

Best Regards,

Community Support Team _ Caiyun

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly. If I misunderstand your needs or you still have problems on it, please feel free to let me know. Thanks a lot!

collinq
Super User
Super User

Hi @rdnguyen ,

 

I have run into this same problem in the past.  There is no "timer" function native to Power BI that would let you daisy-chain the dataflows.  Therefore, the way you are doing it is probably the best way.  It just takes experimentation to get the flow and timing right.  

I have created an idea so please vote for this (and have yoru friends and family vote too) and maybe this will get on Microsoft's radar.

https://ideas.powerbi.com/ideas/idea/?ideaid=e86653d6-adab-ec11-826d-501ac5248c99

 

 




Did I answer your question? Mark my post as a solution!

Proud to be a Datanaut!
Private message me for consulting or training needs.




Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

February 2024 Update Carousel

Power BI Monthly Update - February 2024

Check out the February 2024 Power BI update to learn about new features.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.

Top Solution Authors
Top Kudoed Authors