Forum Discussion
filtering duplicate data across 2 tables
Hi,
I have looked for a solution elsewhere on this forum to my specific problem and cannot find anything that helps, but apologies if anybody knows of a posted solution and please point me in the right direction :)
The problem:
I am working with 3 data tables for this particular requirement. All 3 contain a field called 'store-ID'. The first (and biggest) table contains current year sales data and totals around 750k lines for 2016 YTD. The second table contains a list of stores that have not made any sales of a product in the past week, the third table contains a list of stores that have not sold our product in the past 4 weeks. Tables 2 and 3 contain only unique store ID's, but table 1 contains many repetitions of store ID.
I do not have a definitive list of all store ID's, but this is what I need to solve. The issue is that, although the sales data table will give me most of the stores, there are some that we know have never made a sale, so I need to combine the IDs from table 1 and 2, then remove all duplicates so that I can post a 'Total Stores' metric. I can work this in Excel, but I am struggling with DAX.
Any suggestions would be extremely welcome.
Hi Usates,
After combining the two tables using Appending Queries, to remove the duplicated rows, please go to Remove Rows -> Remove Duplicates.
If you have any question, please feel free to ask.
Best regards,
Yuliana Gu
3 Replies
- ankitpatiraCommunity Champion
acrmorris There is an easy way to do this. Go to power bi desktop query editor -> Under Home tab far right under Combine -> Append Queries as New -> select table 1 and 2 -> Remove unwnated columns except StoreID. You now have store ids from both the tables. Only thing to make sure is before you do this column name for store ids is same for both the tables.
- acrmorrisFrequent Visitor
ankitpatira, thanks for this. I tried this method, but it seems that some duplicates remain. I am expecting the total population of store IDs to be 1,324 but this method leaves me with 1,660 line items (The total of the 'zero sales for 1 week' table is 500, so it seems to work to some extent) . I have exported the query to Excel and applied a simple 'duplicate value' conditional format which throws out the remaining duplicates correctly. Do you have any idea why this might be?
- v-yulgu-msftMicrosoft Employee
Hi Usates,
After combining the two tables using Appending Queries, to remove the duplicated rows, please go to Remove Rows -> Remove Duplicates.
If you have any question, please feel free to ask.
Best regards,
Yuliana Gu