Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
3 years ago

Really need help with dataflow setup

Hello Community:     We have a P1 SKU.     We use dataflows to connect to our ERP system and also to bring in SF data.   The SF dataflows have no issues.     Where I am running into trouble is ...
Need Help
collinq's avatar
collinq
Super User
3 years ago

Hi Anonymous ,

 

My first question is - Why are you refreshing the 2021 and 2022 orders?  Is that data changing?  Shouldn't it be static now?  If so, I would not refresh those two at all.

Then, that leaves the 2023 data.  Are you using incremental refresh?  That would trim the length of time down dramatically.  You don't have to use incremental refresh but not doing so means that it will just get longer and longer refresh times.

Are you appending into a Dataflow that you are using as a dataset or are you using an actual dataset where you are appending the three outputs?  

 

IN a nutshell, I would break this into three separate Dataflows - 2021 Orders and Shipments, 2022 Orders and Shipments, 2023 Orders aned Shipments and then when I have a succesfull 2021 and 2022 I would never refresh those again.   Then, I would either just refresh the 2023 Dataflow (with or without incremental Refresh).  Then, I would have Dataflow 4 wich takes the other 3 dataflows and appends the data and that would be my result set to hit in my Power BI reports.  That means that I only have to worry about Dataflow 3 refresh time and make sure that Dataflow 4 refresh time runs after Dataflow 3 is totally done.