Forum Discussion
Overwrite already existing rows when refreshing
Anonymous
Hi,
I feel that in this scenario removing duplicates is a better option to go with as compared to merge table, considering the fact the there is a single souce table.
Rgds,
Vivek
PS: It is good that you are asking questions. That is how we all learn :)
Thank you for the help vivran22. This is a nice article.
However, I want this to be done automatically, once I hit the "refresh" button.
If I have the 2 queries the article says (the original data and the override data), how can I perform the merge and append functions automatically?
Thank you.
Anonymous
Hi,
Ususally when you hit refresh, Power Query will perform all those tasks (which includes merge and append) which has been defined in the Edit Query mode. So, your entire process should be automated will work with the refresh button.
Otherwise, you may share the sample files and I would like to recreate it for the demonstration purpose.
Rgds,
Vivek
Sorry for the late reply. I experimented a little and I realised that what I ask for can be done with the "Remove Duplicates" option. Seems more simple solution than creating 2 separate tables.
However, does that option deletes the rows or just hides them? What do you think?
Thank you
Anonymous
If your question is whether Remove Duplicate will make changes in the source data then the answer is it will hide the rows. "Remove Duplicates" will keep the first row of the duplicates row and removes/hides the rest before loading to Power BI desktop.
Hope it answers your query.
Rgds,
Vivek
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
- vivran22
My question is if I remove the duplicates, is the size of the data table (or the space it takes) reduced or does it remain the same?
For example if I open a query in power BI with a 10 million row table and then open the same query but now with only 100 rows because the rest are hidden, will they take the same time to load?
Thank you.