Forum Discussion

jjmauras's avatar
jjmauras
Advocate II
10 years ago
Solved

Remove or Hide Duplicate Values

Is there a way to Remove or Ignore duplicate values when they occur in a column?   I'm pulling data from our SQL Server. I have 2 related tables. Every once in a while a duplicate value will occur ...
  • Greg_Deckler's avatar
    Greg_Deckler
    10 years ago

    There is a Remove Duplicates step when editing your query, it is in the Reduce Rows section of the ribbon, the M code is:

    #"Removed Duplicates" = Table.Distinct(#"[Previous Step]", {"[Column Name]"})

jjmauras's avatar
jjmauras
Advocate II
10 years ago

Thanks everyone. Greg_Deckler I think I'm going to go with this solution for the moment as it worked right away. My question is will it stick? For example is it a one time thing or will it remove duplicates every time the SQL data is refreshed?

Greg_Deckler's avatar
Greg_Deckler
Community Champion
10 years ago

Yes, query steps stick. When you run a query, it is really just executing a series of steps. You can see the steps recorded on the righthand side. of the query editing window.

 

Handling this in the source is great, but not always possible. As an end-user focused tool, it is not reasonable to expect that users will have access to the backend to make changes, write stored procedures, expert knowledge to write SQL, etc.