Forum Discussion
Coordinating data flows and PBI loads
What you would need to do, is to schedule them in the order as you have shown?
- Anonymous6 years agoNot applicable
Hi GilbertQ , that's not really a very workable solution in our case. I'm really looking for something dependancy based vs. just guessing / hoping how far apart to schedule jobs so that things like jobs running long or not being kicked off on time in the Power BI service (which happens ALL the time...I often get jobs running a full hour behind) don't hose my dependencies and cause data to be incorrect.
Matthew Roche goes to great length to expound the virtuals of dataflows, so I assumed as a complete dataflow newbie that I was missing something obvious. But I'm sensing from your response that I'm not, and that basic functionality like job dependencies isn't available.
I've thought of combining my 3 "ingest" dataflows into a single one, which would eliminate the issue of trying to make sure 3 input dataflows finish before the compute dataflow refreshes. But the performance in the web is so bad when you start getting many entities in a single dataflow that I've deliberately avoided that...because it can take a good 15 to 30 minutes for a dataflow to do whatever it's "evaluating entities to make sure the load will work properly" step to complete so the job will save. Plus, although this would help get the dataflows synchronized, it still doesn't I think help with kicking off the Power BI dataset refresh once the dataflows are complete.
Maybe I need to look into Power Automate...not sure if that would allow this or not...
Thanks,
Scott