Forum Discussion
vgeldbr
2 years agoHelper IV
Power Query import of CSV file creates duplicate rows
I have a CSV file with just over 13,000 rows. It is an export from a system and is stored in a SharePoint library from where Power BI imports it. The query looks like this:
let
Source = SharePoint.Files("https://sites.company.com/sites/sitename", [ApiVersion = 15]),
#"Filtered Rows" = Table.SelectRows(Source, each [Folder Path] = "https://sites.company.com/sites/sitename/foldername/subfoldername/"),
#"Filtered Rows1" = Table.SelectRows(#"Filtered Rows", each Text.StartsWith([Name], "FileName.csv")),
Navigation = #"Filtered Rows1"{[Name = "FileName.csv", #"Folder Path" = "https://sites.company.com/sites/sitename/foldername/subfoldername/"]}[Content],
#"Imported CSV" = Csv.Document(Navigation, [Delimiter = ",", QuoteStyle = QuoteStyle.Csv]),
#"Promoted headers" = Table.PromoteHeaders(#"Imported CSV", [PromoteAllScalars = true])
in
#"Promoted headers"The CSV file has no duplicate records (based on column [Instance ID] but when Power Query important it the rows are duplicated.
I've spent time googling this and duplicate rows from CSV imports seems to happen a lot not just in Powe Query but I've not found any solution or reason.
If I save just the header row and a single row in a new CSV file this does not happen. Note the issue happens both in Power Query Online (dataflow) and Power BI desktop (December 2023 release).
Does anybody have any ideas? Unfortunately I cannot share the CSV file.