Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get Fabric Certified for FREE during Fabric Data Days. Don't miss your chance! Request now

Reply
Anonymous
Not applicable

Removing duplicates based on Date

I am facing an issue with how i am removing the duplicates from the below set of data in Power Query 

 

What i am trying to do is pick the last price change transaction from a set of data. In the below set, i want the "10004-7/26/2021 2:31:23 PM" to be picked. I tried to sort on key data & time and then removed duplicates on Goods ID and Date. But, for some reason i always ended up with the transaction with an earlier time stamp "10004-7/26/2021 2:12:49 PM"

 

Any better way to resolve this? 1 goodsId can have multiple price change in a day. need to pick the latest price change for that day. 

This works fine when i try to take this below 4 into a table and sort and remove duplicates. But with a bigger dataset behaves different. 

 

ChangeIdGoods IddtDateGoods ID and DateKey date & time
326330100047/26/2021 14:1210004-2021-07-2610004-7/26/2021 2:12:49 PM
326345100047/26/2021 14:3110004-2021-07-2610004-7/26/2021 2:31:23 PM
328262100047/30/2021 14:0710004-2021-07-3010004-7/30/2021 2:07:38 PM
331467100048/10/2021 15:2610004-2021-08-1010004-8/10/2021 3:26:56 PM
1 ACCEPTED SOLUTION
amitchandak
Super User
Super User

@Anonymous , Create s separate data column

date only = DateTime.Date([Date])

 

And the use group by and take max on of the columns that are not needed in group by

Share with Power BI Enthusiasts: Full Power BI Video (20 Hours) YouTube
Microsoft Fabric Series 60+ Videos YouTube
Microsoft Fabric Hindi End to End YouTube

View solution in original post

2 REPLIES 2
ryan_mayu
Super User
Super User

@Anonymous 

just like what @amitchandak  suggest.

you can do group by in PQ

1.PNG





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!




amitchandak
Super User
Super User

@Anonymous , Create s separate data column

date only = DateTime.Date([Date])

 

And the use group by and take max on of the columns that are not needed in group by

Share with Power BI Enthusiasts: Full Power BI Video (20 Hours) YouTube
Microsoft Fabric Series 60+ Videos YouTube
Microsoft Fabric Hindi End to End YouTube

Helpful resources

Announcements
Fabric Data Days Carousel

Fabric Data Days

Advance your Data & AI career with 50 days of live learning, contests, hands-on challenges, study groups & certifications and more!

October Power BI Update Carousel

Power BI Monthly Update - October 2025

Check out the October 2025 Power BI update to learn about new features.

FabCon Atlanta 2026 carousel

FabCon Atlanta 2026

Join us at FabCon Atlanta, March 16-20, for the ultimate Fabric, Power BI, AI and SQL community-led event. Save $200 with code FABCOMM.

Top Solution Authors
Top Kudoed Authors