Forum Discussion
Remove duplicate rows when refreshing
Hi, my issue is that I have some workbooks that contain data from 45 days back each one, and I'll be adding a new workbook weekly, then I have entire rows duplicated with same data in every column, so I need to remove duplicates every time I add a new workbook to the folder and refresh the table. In query Editor in "transform file sample" I add the step "remove duplicates" but it's not working, I tried selecting every column, also in the table icon but doesn't work neither. The only way to make it work is selecting the table and then remove duplicate rows, but i'll have to do that every time I add a file
Hi, Anonymous
Based on your description, I'd like to suggest you refer to Referencing Power Query queries about reference. You may parameterise a data source to load the entire folder. For further information, please refer to the following links.
https://www.biinsight.com/power-bi-desktop-query-parameters-part-1/
https://powerbi.microsoft.com/en-us/blog/deep-dive-into-query-parameters-and-power-bi-templates/
Best Regards
Allan
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
4 Replies
Sounds like you are using some sort of Append query or perhaps a Folder query. Can you create another query, reference your Append or Folder query and add your remove duplicate rows there?
Hi, Anonymous
Based on your description, I'd like to suggest you refer to Referencing Power Query queries about reference. You may parameterise a data source to load the entire folder. For further information, please refer to the following links.
https://www.biinsight.com/power-bi-desktop-query-parameters-part-1/
https://powerbi.microsoft.com/en-us/blog/deep-dive-into-query-parameters-and-power-bi-templates/
Best Regards
Allan
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
