Keeping old data and adding new data on refresh

[RonaldvdH]

Hey Guys,

I have a report that automatically refreshes based on a database but iI want to keep the old data so I can track progress.

The update overwrites the current data so it's always a copy of the realtime info.

Is it possible to create a new table which adds the updated data ?

So if i have table A with 100 adresses in it that will be copied to table B

When Table A updates those 100 adresses the new data will be added to table B but NOT overwrite it (so basically table B would have 200 rows)

When the data updates again, table A will still have the most recent data of those 100 adresses but again those 100 adresse will be added to table B and therefore Table B would have 300 adresses and so on

[AmiraBedh · Accepted solution]

Have you heard before about incremental refresh ?

Incremental refresh extends scheduled refresh operations by providing automated partition creation and management for semantic model tables that frequently load new and updated data. For most models, one or more tables contain transaction data that changes often and can grow exponentially, like a fact table in a relational or star database schema. An incremental refresh policy to partition the table, refreshing only the most recent import partitions, and optionally using another DirectQuery partition for real-time data can significantly reduce the amount of data that has to be refreshed. At the same time, this policy ensures that the latest changes at the data source are included in the query results.

With incremental refresh and real-time data:

• Fewer refresh cycles for fast-changing data are needed. DirectQuery mode gets the latest data updates as queries are processed, without requiring a high refresh cadence.
• Refreshes are faster. Only the most recent data that has changed needs to be refreshed.
• Refreshes are more reliable. Long-running connections to volatile data sources aren't necessary. Queries to source data run faster, reducing potential for network problems to interfere.
• Resource consumption is reduced. Less data to refresh reduces overall consumption of memory and other resources in both Power BI and data source systems.
• Large semantic models are enabled. Semantic models with potentially billions of rows can grow without the need to fully refresh the entire model with each refresh operation.
• Setup is easy. Incremental refresh policies are defined in Power BI Desktop with just a few tasks. When Power BI Desktop publishes the report, the service automatically applies those policies with each refresh.

 https://learn.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-overview