Forum Discussion
Historical data
I'm sorry but as I said I can't use incremental refresh because it has two big limits, it needs an active connection and it refreshes data everytime you operate. For example if I have a value column and I create a new column that returns [value]+1 it doesn't use stored data but pretends to refresh all the data from the start.
AGo well you have to make sure you know what columns you need. It is like traditionally building a warehouse and after 10 years of data, if you need to add new column, you have to backfill your warehouse and similarily, if you are adding new columns etc, ofcourse you have to update your dataset (and again full refresh will be one time) and then it will be inremental refresh. Not sure what you expect the solution suppose to be if incremental refresh is not the right approach here.
Also I'm not sure what you mean by active connection?? Data refresh required connection.
yes, the dataset will never change and I need to add new columns but not from the DB but calculated from the dataset itself. So there's no reason for refresh the dataset. It seems that PowerBI needs to refresh data for every change in derived data also if the fundamental data are the same. There is no sense in pretending to refresh all the historical data if I'd like to add a column like valueA*valueB, PowerBI has all the information to operate this multiplication. Most of the time the limit of 2 hours for the first refresh in incremental refresh is too low, using PowerBI desktop to query the large amount of data is the most efficient way too freeze it.
Maybe someone can answer with a solution to keep historical data freezed and allowing to operate like we do in dax but directly in power query.
