Forum Discussion
filtering on multiple columns
- Anonymous1 year ago
Hi wynhodgkiss,
Thank you for reaching out to the Microsoft Fabric Forum Community.
You can try the following Power Query code to achieve your goal(Modify according to your requirement).
Source = Folder.Files("C:\Users\YourName\OneDrive\YourFolderPath"),
FilteredCSVs = Table.SelectRows(Source, each Text.EndsWith([Extension], ".csv")),
Imported = Table.AddColumn(FilteredCSVs, "Content", each Csv.Document(File.Contents([Folder Path] & [Name]), [Delimiter=",", Columns=3, Encoding=65001, QuoteStyle=QuoteStyle.None])),
Expanded = Table.ExpandTableColumn(Imported, "Content", {"Column1", "Column2", "Column3"}, {"WeekRef", "Index", "Name"}),// Change data types
ChangedTypes = Table.TransformColumnTypes(Expanded, {{"WeekRef", type text}, {"Index", Int64.Type}, {"Name", type text}}),// Sort by Index (ascending)
Sorted = Table.Sort(ChangedTypes, {{"Index", Order.Ascending}}),// Remove duplicates based on WeekRef
Deduplicated = Table.Distinct(Sorted, {"WeekRef"}),// Select final columns
Final = Table.SelectColumns(Deduplicated, {"WeekRef", "Index", "Name"})
in
FinalAs a side note, is there any possibility that a newer file could have a higher Index due to sync delays or upload timing? If so, how would you suggest handling that edge case?
If this solution helped, please consider marking the response as accepted and giving it a thumbs-up so others can benefit as well.
Best regards,
Prasanna Kumar
Hi wynhodgkiss,
Thank you for reaching out to the Microsoft Fabric Forum Community.
You can try the following Power Query code to achieve your goal(Modify according to your requirement).
Source = Folder.Files("C:\Users\YourName\OneDrive\YourFolderPath"),
FilteredCSVs = Table.SelectRows(Source, each Text.EndsWith([Extension], ".csv")),
Imported = Table.AddColumn(FilteredCSVs, "Content", each Csv.Document(File.Contents([Folder Path] & [Name]), [Delimiter=",", Columns=3, Encoding=65001, QuoteStyle=QuoteStyle.None])),
Expanded = Table.ExpandTableColumn(Imported, "Content", {"Column1", "Column2", "Column3"}, {"WeekRef", "Index", "Name"}),
// Change data types
ChangedTypes = Table.TransformColumnTypes(Expanded, {{"WeekRef", type text}, {"Index", Int64.Type}, {"Name", type text}}),
// Sort by Index (ascending)
Sorted = Table.Sort(ChangedTypes, {{"Index", Order.Ascending}}),
// Remove duplicates based on WeekRef
Deduplicated = Table.Distinct(Sorted, {"WeekRef"}),
// Select final columns
Final = Table.SelectColumns(Deduplicated, {"WeekRef", "Index", "Name"})
in
Final
As a side note, is there any possibility that a newer file could have a higher Index due to sync delays or upload timing? If so, how would you suggest handling that edge case?
If this solution helped, please consider marking the response as accepted and giving it a thumbs-up so others can benefit as well.
Best regards,
Prasanna Kumar
- wynhodgkiss1 year agoAdvocate II
My initial thought was to use sorting to achieve this but it didn't quite work as expected. Removing duplicates didn't remove the correct duplicates if that makes sense.
- wynhodgkiss1 year agoAdvocate II
I think this is the solution, just needed an additional sort on the WeekRef.