Forum Discussion

danilomorganti's avatar
danilomorganti
Frequent Visitor
5 years ago

Incremental refresh on data flow

Hello everyone! I have a problem with the incremental update of the data stream.
I have successfully set up incremental update and scheduled update once a day.
If I manually update the flow, the incremental update works fine, otherwise the scheduled update will overwrite all the data and delete the history.


the questions are:
-the incremental update how often is it performed?
- is it correct to set both incremental and scheduled update on the same dataset?
- is there a way to perform incremental update automatically?

Thanks!!

Need Help

2 Replies

  • lbendlin's avatar
    lbendlin
    Super User
    5 years ago

    You know that dataflows is basically a bunch of CSV files, right? Incremental refresh is not really what you do with dataflows.

  • danilomorganti's avatar
    danilomorganti
    Frequent Visitor
    5 years ago

    I get data from web API, at first time I get the data from 2019 to now but each time data are refreshed the service overwrites all the data and i want it.
    I would like to streamline the API and every day add and overwrite ognly the last 7 day.

     

    How can I do it? How can I "frozen" the history data and add / overwrite only the last 7 days of data every day?

     

    Thanks!