integrating update

[gunicotra]

Hi to all,

everyday I update my source table with fresh data. This procedure overrides the previous and I wish I can append new daily data to the existing table in order for old data not to be over overwritten. I tried with:

- "incremental refresh": but this precedure overrides anyway the data which have been changed and it is not what I am searching for;

- append manually the daily data: i.e.: manually copying olda data into a new table, then refresh table, then appending new data into the manually-copied-table. But, frankly speaking, I am sure there must be a smarter and automatic procedure which avoids me to manually do all this stuff every day!!

Can you pleas give me some hints?

Thanks

G

[v-jingzhang]

Hi @gunicotra 

You need to store historical data at the data source side. What is your data source? Maybe you can consider use some automatic tools or task scheduler tools to copy the old data into a new location automatically every day. For example, with Power Automate, we can create an automatic flow to copy a file and store the copy with a new name in another location. Then connect to all tables and combine them with Power BI. 

Best Regards,
Community Support Team _ Jing

If this post helps, please Accept it as Solution to help other members find it.

[gunicotra]

My data source is a web sites where those data are published daily. However for me it is important to have the historical data thus I need to collect yesterdays values, and then append them to todays data. 
Moreover data flows at powerquery-transformation level thus I have not much room to act and model data so to activate a power automate process

[parry2k]

@gunicotra you cannot keep the data if your source data doesn't have an append option. You need to append data at the source and there is nothing you need on the Power BI side. Whenever refresh is done in Power BI, it is pretty much a full refresh, so if your source has only new data, nothing can be done.

✨ Follow us on LinkedIn

Check my latest blog post The Power of Using Calculation Groups with Inactive Relationships (Part 1) (perytus.com) I would ❤ Kudos if my solution helped. 👉 If you can spend time posting the question, you can also make efforts to give Kudos to whoever helped to solve your problem. It is a token of appreciation!

⚡ Visit us at https://perytus.com, your one-stop-shop for Power BI-related projects/training/consultancy.⚡