Forum Discussion
Post PatronHistorical data
Super UserAGo well you have to make sure you know what columns you need. It is like traditionally building a warehouse and after 10 years of data, if you need to add new column, you have to backfill your warehouse and similarily, if you are adding new columns etc, ofcourse you have to update your dataset (and again full refresh will be one time) and then it will be inremental refresh. Not sure what you expect the solution suppose to be if incremental refresh is not the right approach here.
Also I'm not sure what you mean by active connection?? Data refresh required connection.
Post Patronyes, the dataset will never change and I need to add new columns but not from the DB but calculated from the dataset itself. So there's no reason for refresh the dataset. It seems that PowerBI needs to refresh data for every change in derived data also if the fundamental data are the same. There is no sense in pretending to refresh all the historical data if I'd like to add a column like valueA*valueB, PowerBI has all the information to operate this multiplication. Most of the time the limit of 2 hours for the first refresh in incremental refresh is too low, using PowerBI desktop to query the large amount of data is the most efficient way too freeze it.
Maybe someone can answer with a solution to keep historical data freezed and allowing to operate like we do in dax but directly in power query.
- parry2k
AGo if you add new columns as DAX expression you don't need to refresh the dataset, it is always added on existing dataset. You need to know basic fundamentals , difference betwee adding column using power query or getting new columns from datasource vs calculating column using DAX. Wiht DAX, it works on existing loaded data.
- AGoYes I know, but unfortunately some functions for derived data are not available in dax (like intelligent unpivoting and row number indexing without ranking). That's why I need to literally freeze historical data besides that it's impossible for me to query historical data in 2 hours limit or mandatory refreshing datasource everytime I apply changes on derived data functions. I hope there's a solution.