Forum Discussion

Oplink's avatar
Oplink
New Member
6 years ago
Solved

Identifying and filtering duplicates

I have tryed to solve this with DAX on/off for a couple of weeks now with limited success.

 

I have a large table that each day is filled with subscription data by a script, sometimes the script runs twice, and create duplicate data with different timestamps (DateTime collum).

I wish to identify the duplicates and filter them out in my visuals.

 

My idea was to first identify if there was a duplicate entry on a given date by counting SubscriptionID, since this should be uniqe each day, adding a True/False collum called "Have Duplicate". Then I would add a True/False collum called "Is Duplicate" determing this using the "Have Duplicate" collum and setting all entries after the first DateTime as True.

 

I can do a count on SubscriptionID just fine, idetifing if there is duplicates, but as soon as I try to use DateTime to set the "Is Duplicate" it breaks.

 

I have a DateTable that is related to the Date collum, and I suspect that it is the one breaking it, buy have run out of ideas to solve this.

 

Best Regards

Ole

  • Anonymous's avatar
    Anonymous
    6 years ago

    Did you tried selecting all the columns except the timestamp column? Do you maybe have some sample data? 

4 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    If you want to remove the duplicates, you can achieve this using Power Query. 

    Select all the columns except the date column, click Remove Rows > select "Remove Duplicates"

     

    • Oplink's avatar
      Oplink
      New Member

      It was the first thing I tried, but since they are not really duplicates, just duplicate data, with different timestamps (DateTime), havn't I been able to get that approach to work for me. 

       

      I might be doing something wrong ?

       

      • Anonymous's avatar
        Anonymous
        Not applicable

        Did you tried selecting all the columns except the timestamp column? Do you maybe have some sample data?