scheduling ingestion dataflow, transformation dataflow, then final dataset

[frano72]

Hi, lets say i have an ingestion dataflow that takes 5 mins to run, then a transformation dataflow that has its source as the ingestion one (and it takes 5 mins to run) and then lastly a dataset that has its source as the transformation one (and it takes 5 mins to run).

Now, if you could somehow chain these together - it would take a total of 15 mins.

If you use the interface to schedule these refreshes, you have to keep to 30 min time boundaries so am i right that the quickest you could get these synched is 65 mins ?

Ingestion at time =0

Transforamtion at time = 30 mins

Data set at time = 60 mins (and then takes 5 mins to complete).

(this was not meant to sound like one of those facebook adding up quizzes...)

sb

[v-lid-msft]

Hi @frano72 ,

Based on my refresh, we can only set scheduled refresh in 00/30 minutes under shared capacity, also notice that,  for the configured scheduled refresh, the target is to initiate the refresh within 15 minutes of the scheduled time slot, but a delay of up to one hour can occur if the service can't allocate the required resources sooner (30 mintues between each refresh should enough). So if set scheduled refresh for those dataflow and dataset, it may take 30+30+30+5=65 minutes or longer to complete entire refresh process,  if you want to ensure it can refresh success. So it seems combine some of them will be a greate idea. But we can also use Refresh Dataflow API and Refresh Dataset API (or this for app workspace) to trigger on-demand refresh at any time and execute right after api call, but notice that  In Shared capacities this call is limited to eight times per day (including refreshes executed via Scheduled Refresh)

Best regards,