Forum Discussion
Streaming Dataset or Dataflow to store historical data ?
Hi there
I would like to store historical data on a daily basis to make trend KPI and I wonder which technology is the more relevant. Note that the old data are removed from the source (an SQL database with a custom OLEDB connector) and I don't really see if it can be stored into the dataflow or in the streaming dataset. My data model is not straighforward, and I don't think that simple json structures used in the streaming dataset could fit, so I guess dataflow should be the solution, however in all the examples I saw, the data are not deleted from the source, so I can't figure out what to do with Power BI service.
- Hi there
If your source data is being deleted you could potentially store it in a dataflow. But if you ever needed to refresh the data you would loose it because the historical data is not there.
What I would suggest is to rather store the data from your original source into another database to keep the history.
And then load from the historical database?
4 Replies
- GilbertQ
Super User
Hi there
If your source data is being deleted you could potentially store it in a dataflow. But if you ever needed to refresh the data you would loose it because the historical data is not there.
What I would suggest is to rather store the data from your original source into another database to keep the history.
And then load from the historical database?- GilbertQThanks for letting us know.