Forum Discussion
Streaming Dataset or Dataflow to store historical data ?
Hi there I would like to store historical data on a daily basis to make trend KPI and I wonder which technology is the more relevant. Note that the old data are removed from the source (an SQL...
- Hi there
If your source data is being deleted you could potentially store it in a dataflow. But if you ever needed to refresh the data you would loose it because the historical data is not there.
What I would suggest is to rather store the data from your original source into another database to keep the history.
And then load from the historical database?
GilbertQ
Super User
Super UserHi there
If your source data is being deleted you could potentially store it in a dataflow. But if you ever needed to refresh the data you would loose it because the historical data is not there.
What I would suggest is to rather store the data from your original source into another database to keep the history.
And then load from the historical database?
If your source data is being deleted you could potentially store it in a dataflow. But if you ever needed to refresh the data you would loose it because the historical data is not there.
What I would suggest is to rather store the data from your original source into another database to keep the history.
And then load from the historical database?
Well if you are right, this is indeed the best solution.
- GilbertQThanks for letting us know.
Well as I can get the data from my sql source, I will create a scheduled task to extract them every week and store them in another database with a new "week number" column. It is the simpler solution.