Forum Discussion
How to remove duplicates based on sort order
My suggestion would be the following, illustrated by this video.
I created some test data in Access just to verify if the code would allow for Query Folding (by rightclicking each step in the queries and check for the existence of Native Query). They all did, so performance should be OK..
I created a query InputData that just reads the data from Access. For this query, load is disabled.
let
Source = Access.Database(File.Contents("C:\Users\Marcel\Documents\Forum bijdragen\Power BI Community\How to remove duplicates based on sort order.accdb"), [CreateNavigationProperties=true]),
_Requests = Source{[Schema="",Item="Requests"]}[Data]
in
_Requests
I created a query MinStatusByRequestID with input from query InputData that determines the minimum statusID for each RequestID using Group By on the Transform tab. Also for this query, load is disabled.
let
Source = InputData,
#"Grouped Rows" = Table.Group(Source, {"RequestID"}, {{"minStatus", each List.Min([statusID]), type number}})
in
#"Grouped Rows"
I created a query FilteredData that merges the previous 2 queries with an inner join. For this query, load is enabled.
let
Source = InputData,
#"Merged Queries" = Table.Join(Source,{"RequestID", "statusID"},MinStatusByRequestID,{"RequestID", "minStatus"},JoinKind.Inner),
#"Removed Columns" = Table.RemoveColumns(#"Merged Queries",{"minStatus"})
in
#"Removed Columns"
Thank you so much Marcel! I have been struggling for days and googling crazy for a solution and you just solved my problem!!!