Forum Discussion
Removing duplicates based on a sort order
Hey folks I am trying to get rid of duplicate data and want to keep the record that was last modified.
I added a sort but the remove duplicate transformation seems to ingore it.
I found the solution here in case anyone else needs it.
https://community.powerbi.com/t5/Desktop/Remove-duplicates-based-on-sort-in-Power-Query/m-p/592660#M281250
4 Replies
- ryan_mayuSuper User
add a new step in PQ
= Table.Group(Source,"REVENUE STREAM",{"data",each Table.Max(_,"LASTMODIFYDATE")})
change the revenue stream to the column that identify the duplicates
then expand the data
pls see the attachment below
- Ashish_MathurSuper User
Hi,
Share complete information. What do you want to group by? - revenue stream or some other column?
- YosinatorFrequent Visitor
Thank you for responding guys!
Just to be clear there are several instances for duplicated records in the table.
Will this solution work across the board?
All the duplicates share the same values in the "hs_object_id" column. - YosinatorFrequent Visitor
I found the solution here in case anyone else needs it.
https://community.powerbi.com/t5/Desktop/Remove-duplicates-based-on-sort-in-Power-Query/m-p/592660#M281250