Forum Discussion

acrmorris's avatar
acrmorris
Frequent Visitor
9 years ago
Solved

filtering duplicate data across 2 tables

Hi,

 

I have looked for a solution elsewhere on this forum to my specific problem and cannot find anything that helps, but apologies if anybody knows of a posted solution and please point me in the right direction :)

 

The problem:

I am working with 3 data tables for this particular requirement. All 3 contain a field called 'store-ID'. The first (and biggest) table contains current year sales data and totals around 750k lines for 2016 YTD. The second table contains a list of stores that have not made any sales of a product in the past week, the third table contains a list of stores that have not sold our product in the past 4 weeks. Tables 2 and 3 contain only unique store ID's, but table 1 contains many repetitions of store ID.

 

I do not have a definitive list of all store ID's, but this is what I need to solve. The issue is that, although the sales data table will give me most of the stores, there are some that we know have never made a sale, so I need to combine the IDs from table 1 and 2, then remove all duplicates so that I can post a 'Total Stores' metric. I can work this in Excel, but I am struggling with DAX.

 

Any suggestions would be extremely welcome.

  • acrmorris

    Hi Usates,

     

    After combining the two tables using Appending Queries, to remove the duplicated rows, please go to Remove Rows -> Remove Duplicates.

     

    If you have any question, please feel free to ask.

     

    Best regards,
    Yuliana Gu

3 Replies

  • ankitpatira's avatar
    ankitpatira
    Community Champion

    acrmorris There is an easy way to do this. Go to power bi desktop query editor -> Under Home tab far right under Combine -> Append Queries as New -> select table 1 and 2 -> Remove unwnated columns except StoreID. You now have store ids from both the tables. Only thing to make sure is before you do this column name for store ids is same for both the tables.

    • acrmorris's avatar
      acrmorris
      Frequent Visitor

      ankitpatira, thanks for this. I tried this method, but it seems that some duplicates remain. I am expecting the total population of store IDs to be 1,324  but this method leaves me with 1,660 line items (The total of the 'zero sales for 1 week' table is 500, so it seems to work to some extent) . I have exported the query to Excel and applied a simple 'duplicate value' conditional format which throws out the remaining duplicates correctly. Do you have any idea why this might be?

       

       

  • v-yulgu-msft's avatar
    v-yulgu-msft
    Microsoft Employee

    acrmorris

    Hi Usates,

     

    After combining the two tables using Appending Queries, to remove the duplicated rows, please go to Remove Rows -> Remove Duplicates.

     

    If you have any question, please feel free to ask.

     

    Best regards,
    Yuliana Gu