Remove duplicate rows when refreshing

[Anonymous]

Hi, my issue is that I have some workbooks that contain data from 45 days back each one, and I'll be adding a new workbook weekly, then I have entire rows duplicated with same data in every column, so I need to remove duplicates every time I add a new workbook to the folder and refresh the table. In query Editor in "transform file sample" I add the step "remove duplicates" but it's not working, I tried selecting every column, also in the table icon but doesn't work neither. The only way to make it work is selecting the table and then remove duplicate rows, but i'll have to do that every time I add a file

[Greg_Deckler]

Sounds like you are using some sort of Append query or perhaps a Folder query. Can you create another query, reference your Append or Folder query and add your remove duplicate rows there?

[Anonymous]

hello, i create a reference from my sample file but when I refresh, only load the file that used for the sample, what should I do to use the parameters that a I set and load the entire folder?

[v-alq-msft · Accepted solution]

Hi, @Anonymous 

Based on your description, I'd like to suggest you refer to Referencing Power Query queries about reference. You may parameterise a data source to load the entire folder. For further information, please refer to the following links.

https://www.biinsight.com/power-bi-desktop-query-parameters-part-1/

https://powerbi.microsoft.com/en-us/blog/deep-dive-into-query-parameters-and-power-bi-templates/

Best Regards

Allan

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous]

yes I loaded a folder and have to append to combine multiple worksheets of each workbook. taking all the data to a new query/table will resolve this issue? Sorry i'm new in this. There's no way to remove it directly in the created table? 
These are the steps applied in the sample file