Forum Discussion
Remove duplicates, keep the latest entry (Sharepoint as source)
Hi! I have here a FCONS table that has multiple Reservation No.
I only need the LATEST entry of the Reservation No. (highlighted in yellow)
I have resolved this before with Table.Buffer on my local folder as source. But since I have migrated the source to Sharepoint, I need to redevelop this again and noticed that Table.Buffer may not be applicable anymore since it slows down the performance.
Hoping someone could help me with a workaround, Or walk me through the Table.Buffer with Sharepoint as source.
EDIT: As much as possible, the most EFFICIENT way applicable to BIG DATA.
Thanks!
Hi! Thanks for taking time. I have resolved the issue by sorting descendingly the RunDate then partition by Reservation No. - add index column, expand the partition table and remove its duplicates.
7 Replies
Hi! Thanks for taking time. I have resolved the issue by sorting descendingly the RunDate then partition by Reservation No. - add index column, expand the partition table and remove its duplicates.
Hi hmenco ,
not sure if this would be faster, but it might be worth a try:
Group by reservation and cust no and add 2 fields:
max Rundate and All
Expand All and filter Rundate = MaxRundateAlthough the grouping operation will cost performance, no need to sort and buffer.
lbendlin ,
the sort order you've described is not guaranteed to stick:Bug warning for Table.Sort and removing duplicates... - Microsoft Power BI Community
There's a simple trick. Sort your data as needed (which is unfortunate as it is very costly) - in your case you need to sort by Rundate descending - and then select the {Reservation No.] column and choose "Remove Duplicates". That will only keep the first row for each Reservation No. which is actually the last row due to the sorting etc.
I've done this but it doesn't keep the LATEST entry. Any other workaround? Thanks!
