Append Historical data with current data without overwriting

[abbas_uddin]

Dear Experts,

i am in a problematic situation related to historical data and new data. i will explain the scenario below:

we have a historical data which we will load till Sep 2021 one time (through excel files). The current data is imported through web service. Now question is how can we combine historical data with current data without overwriting the data. Once the current month is finished, next month will overwrite the previous month and previous month will not be copied in historical data (as it is loaded manually till Sep 2021 one time only).

The current data is always month to date. 

is there any incremental data update where it copies the current data in historical?. we cannot run more than one month data in webservice as it timesout the system. So either we copy daily or when month is finished. 

Objective is not lose any data. kindly your input needed. Thanks alot

[Anonymous]

Hi @abbas_uddin ,

As the similar thread mentioned:

You may refer to:

Storing and using information from a dynamic data source using PBI desktop.

Historical Data Preservation using Power BI Dataflow

Keep The Existing Data In Your Power BI Dataset And Add New Data To It Using Incremental Refresh

Best Regards,
Eyelyn Qin
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[abbas_uddin]

@mahoneypat is there any suggestion how do we get the export working? so you mean to say we export the most recent month or current month data and export it as soon as the month is finished. Our objective is simple, current data should get added to historical data as soon as the month or day is finished, whatever way is fine. 

[mahoneypat]

The simplest way would be to use DAX Studio at the end of each month to export the month of data as CSV or Excel.  You can then store the monthly files on SharePoint (for example) and then append your original data, the monthly files to date, and the current month REST API data.

Pat

[mahoneypat]

You may need to export the data for your most recent month and append it to both your historical Sep2021 data and your current month refresh.

Pat