Forum Discussion
Incremental refresh with no parameters advice
Hi All,
I'm new to setting up incremental refreshes and I have read through microsoft's and other's instruction for setting these up. However in my case I have daily data being dropped to my gateway folder that over writes the previous days data. So in my refresh, I just need it to grab the entire data set and add it to my report data. I can not find a way to set it up with out appling a parameter. Am I missing something or would you have any advice or links to instruction I'm unable to find that might help?
Thanks as always for any help you can offer.
Terry
- Hi there
Even though you set it to only go back one day, because it cannot do query folding it has to read all the data to find the last day.
Incremental refreshing is the right approach but it only works on data sources that can do query folding which is typically database systems
12 Replies
- Hi there
If you are always getting the entire dataset, then why would you need incremental refresh?
Unless you mean that you want to load the new data and keep the existing data in your model.
If so you would need to use the RangeStart and RangeEnd columns. And as far as I know I do not think Incremental refresh is supported for local files such as CSV or Excel file.Thanks Gilbert!
Yes I do want each new days data to be added be my existing data in my model. I'll try yuor suggestion, since I have no defined range, is there a wildcard indicator I can use to grab all?
Terry
- Hi there
Currently you would have to use the RangeStart and RangeEnd which have to be DateTime columns.
Hey tbobolz , sorry for barging in,
I have a very similar requirement - a 'point in time' data that I need to keep adding rather than overwriting upon refresh (in order to create trends).
I couldn't understand if you actually managed to do this using the Incremental Refresh feature.
If you did, I'd be happy if you could elaborate a bit on your solution.
Thanks a lot!Hi Yaa_D
No this was ot achievable on a csv file, and I moved to doing a simple refresh of the entire data set each day, as it is not that large. Also, I can control my data set through the software I am exporting it from. Sorry, probably not the answer you were looking for.
I do need to try this again on another tool I am developing, but as suggested in this conversation, it does not work with csv files. I'll need to use an SQL export and have IT help me do the initial set-up.
Good luck on your project!
