Forum Discussion

wynhodgkiss's avatar
wynhodgkiss
Advocate II
1 year ago
Solved

filtering on multiple columns

Hi, I have a really simple table of: FileName, Index (from 1 with 1 being the latest received file) and WeekRef (6 digit text that contains the weeks in each file). WeekRef will duplicate as each f...
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi wynhodgkiss,

     

    Thank you for reaching out to the Microsoft Fabric Forum Community.

     

    You can try the following Power Query code to achieve your goal(Modify according to your requirement).

     

    Source = Folder.Files("C:\Users\YourName\OneDrive\YourFolderPath"),
    FilteredCSVs = Table.SelectRows(Source, each Text.EndsWith([Extension], ".csv")),
    Imported = Table.AddColumn(FilteredCSVs, "Content", each Csv.Document(File.Contents([Folder Path] & [Name]), [Delimiter=",", Columns=3, Encoding=65001, QuoteStyle=QuoteStyle.None])),
    Expanded = Table.ExpandTableColumn(Imported, "Content", {"Column1", "Column2", "Column3"}, {"WeekRef", "Index", "Name"}),

    // Change data types
    ChangedTypes = Table.TransformColumnTypes(Expanded, {{"WeekRef", type text}, {"Index", Int64.Type}, {"Name", type text}}),

    // Sort by Index (ascending)
    Sorted = Table.Sort(ChangedTypes, {{"Index", Order.Ascending}}),

    // Remove duplicates based on WeekRef
    Deduplicated = Table.Distinct(Sorted, {"WeekRef"}),

    // Select final columns
    Final = Table.SelectColumns(Deduplicated, {"WeekRef", "Index", "Name"})
    in
    Final

     

    As a side note, is there any possibility that a newer file could have a higher Index due to sync delays or upload timing? If so, how would you suggest handling that edge case?

     

    If this solution helped, please consider marking the response as accepted and giving it a thumbs-up so others can benefit as well.

     

    Best regards,
    Prasanna Kumar