Forum Discussion
johnmcauliffe
3 years agoNew Member
DataFormat.Error: We couldn't Convert to Number - But it's all Text and I haven't changed type
Please Help. I'm getting this error when filtering after a merged query. I can locate the error in the source data - which is a large .csv file. The column is mostly numbers but has some text values....
johnmcauliffe
3 years agoNew Member
There are some steps and the use case requires merged queries. The initial import Transform File:
let
Source = Csv.Document(Parameter2,[Delimiter=";", Columns=33, Encoding=1252, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type datetime}, {"SAMPLE COLUMN", type datetime}, {"ERROR GENERATING COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}, {"SAMPLE COLUMN", type text}})
in
#"Changed Type"
Then the Query on the same data:
let
Source = SharePoint.Files("https://Source.sharepoint.com/teams/GO365_SourceSite", [ApiVersion = 15]),
#"Filtered Rows" = Table.SelectRows(Source, each Text.Contains([Folder Path], "Filter Value") and not Text.Contains([Folder Path], "Filter Value")),
#"Filtered Hidden Files1" = Table.SelectRows(#"Filtered Rows", each [Attributes]?[Hidden]? <> true),
#"Invoke Custom Function1" = Table.AddColumn(#"Filtered Hidden Files1", "Transform File (2)", each #"Transform File (2)"([Content])),
#"Renamed Columns1" = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}),
#"Removed Other Columns1" = Table.SelectColumns(#"Renamed Columns1", {"Source.Name", "Transform File (2)"}),
#"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform File (2)", Table.ColumnNames(#"Transform File (2)"(#"Sample File (2)"))),
#"Duplicated Column" = Table.DuplicateColumn(#"Expanded Table Column1", "SAMPLE COLUMN", "SAMPLE COLUMN - Copy"),
#"Duplicated Column1" = Table.DuplicateColumn(#"Duplicated Column", "SAMPLE COLUMN1", "SAMPLE COLUMN1 - Copy"),
#"Merged Columns" = Table.CombineColumns(#"Duplicated Column1",{"SAMPLE COLUMN - Copy", "SAMPLE COLUMN1 - Copy"},Combiner.CombineTextByDelimiter(" ", QuoteStyle.None),"ID"),
#"Changed Type" = Table.TransformColumnTypes(#"Merged Columns",{{"ERROR GENERATING COLUMN", type text}})
in
#"Changed Type"
Then the output query (The error occurs at #"Filtered Rows" step):
let
Source = Table.NestedJoin(#"Client List", {"IDENTIFIERA"}, #"Status", {"IDENTIFIERB"}, "Status", JoinKind.LeftOuter),
#"Expanded Status" = Table.ExpandTableColumn(Source, "Status", {"Code", "SAMPLE COLUMN", "SAMPLE COLUMN", "SAMPLE COLUMN", "SAMPLE COLUMN"}, {"Status.Code", "Status.SAMPLE COLUMN", "Status.SAMPLE COLUMN", "Status.SAMPLE COLUMN", "Status.SAMPLE COLUMN"}),
#"Duplicated Column" = Table.DuplicateColumn(#"Expanded Status", "Account", "Account - Copy"),
#"Duplicated Column1" = Table.DuplicateColumn(#"Duplicated Column", "Status.SAMPLE COLUMN", "Status.SAMPLE COLUMN - Copy"),
#"Merged Columns" = Table.CombineColumns(#"Duplicated Column1",{"Account - Copy", "Status.SAMPLE COLUMN - Copy"},Combiner.CombineTextByDelimiter(" ", QuoteStyle.None),"ID"),
#"Merged Queries" = Table.NestedJoin(#"Merged Columns", {"Status.SAMPLE COLUMN"}, #"QUERY HFA", {"SAMPLE COLUMN"}, "SAMPLE COLUMN HFA", JoinKind.LeftOuter),
#"Expanded HFA" = Table.ExpandTableColumn(#"Merged Queries", "SAMPLE COLUMN HFA", {"SAMPLE COLUMN", "SAMPLE COLUMN TYPE"}, {"HFA.SAMPLE COLUMN", "HFA.SAMPLE COLUMN"}),
#"Merged Queries1" = Table.NestedJoin(#"Expanded HFA", {"HFA.SAMPLE COLUMN"}, #"ERROR QUERY", {"SAMPLE COLUMN"}, "ERROR QUERY", JoinKind.LeftOuter),
#"Expanded ERROR QUERY" = Table.ExpandTableColumn(#"Merged Queries1", "ERROR QUERY", {"SAMPLE COLUMN", "ERROR GENERATING COLUMN"}, {"ERROR QUERY.SAMPLE COLUMN", "ERROR QUERY.ERROR GENERATING COLUMN"}),
#"Added Conditional Column" = Table.AddColumn(#"Expanded ERROR QUERY", "Match Check", each if [ERROR QUERY.ERROR GENERATING COLUMN] = null then "No" else if [ERROR QUERY.ERROR GENERATING COLUMN] = "" then "No" else if [Account] = [ERROR QUERY.ERROR GENERATING COLUMN] then "Yes" else "No"),
#"Filtered Rows" = Table.SelectRows(#"Added Conditional Column", each ([Match] = "Yes")),
#"Merged Queries2" = Table.NestedJoin(#"Filtered Rows", {"Status.Code"}, OTHER QUERY, {"Code"}, "OTHER QUERY", JoinKind.LeftOuter),
#"Expanded OTHER QUERY" = Table.ExpandTableColumn(#"Merged Queries2", "OTHER QUERY", {"SAMPLE COLUMN", "SAMPLE COLUMN", "SAMPLE COLUMN", "SAMPLE COLUMN", "SAMPLE COLUMN", "SAMPLE COLUMN", "SAMPLE COLUMN"}, {"OTHER QUERY.SAMPLE COLUMN", "OTHER QUERY.SAMPLE COLUMN", "OTHER QUERY.SAMPLE COLUMN", "OTHER QUERY.SAMPLE COLUMN", "OTHER QUERY.SAMPLE COLUMN", "OTHER QUERY.SAMPLE COLUMN", "OTHER QUERY.SAMPLE COLUMN"}),
#"Added Conditional Column1" = Table.AddColumn(#"Expanded OTHER QUERY", "Direction Match", each if [Direction] = [#"OTHER QUERY.SAMPLE COLUMN"] then "Yes" else null),
#"Filtered Rows1" = Table.SelectRows(#"Added Conditional Column1", each [Direction Match] <> null and [Direction Match] <> "")
in
#"Filtered Rows1"
I've named the column as ERROR GENERATING COLUMN throughout. The error occurs when the related conditional column is filtered. I cannot see why Power Query is trying to convert this data from the error column to a number at this point?