Help: Saving and aggregating old versions of a dataset

[IgorGershenson]

I have access to a dataset that gets overwritten each time it is refreshed. However, I want to save the older versions of this dataset. Ideally, I want to have one large table that has all the history along with a date corresponding to each version. What is the easiest way to accomplish this? I tried doing it the hard way by importing each day's version into Excel using Power Query. However, I can't load it as a table into a worksheet because it is too big. If I load it as a connection only and then later try to load it into Power BI, PBI tells me the Excel file table is empty. Thanks in advance for your help. 

[Anonymous · Accepted solution]

Hi  @IgorGershenson ,

You can consider using incremental refresh, which allows you to load data in an incremental manner, extending the scheduled refresh operation by providing automatic partition creation and management for dataset tables that frequently load new and updated data, which will dynamically partition and separate data that needs to be refreshed frequently from data that can be refreshed infrequently.

When Power BI Desktop publishes a report to the Power BI service, the service will create incremental refreshes and historical partitions during the first refresh operation, which will add all the data before the date and time when it is used for the first time, which may result in long load times.

Refer to：

Incremental refresh for datasets and real-time data in Power BI - Power BI | Microsoft Learn

Best Regards,

Liu Yang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[amitchandak]

@IgorGershenson , Based on what I got. Check if this can help

https://blog.crossjoin.co.uk/2020/04/13/keep-the-existing-data-in-your-power-bi-dataset-and-add-new-...
https://www.thebiccountant.com/2017/01/11/incremental-load-in-powerbi-using-dax-union/