Forum Discussion
Using append to grow a dataset
So, the behavior you are seeing is the expected behavior. Incremental data load does not exist quite yet. Therefore, the second query in your Append is replacing its data. Unfortunately, at this time you would have to copy your second query each day and add it to your Append query, disabling data load on your previous query. This is currenly the only real way to incrementally grow your data set.
steveJ- Something else that you could do would be to use an R Script step in your query to archive the information into a SQL database and build up an appended data set there. Then you could turn around and ingest this database table into your model. Kind of a convoluted way of doing it but probably no more so than Excel macros.
I'd have to learn R and how to set up an SQL db at that point. A quick Macro should do it. Thanks! hopefully the functionality will exist, should be useful enough to warrant it.