Forum Discussion

joooffice's avatar
joooffice
Helper I
5 years ago
Solved

Matching Data that could be in different columns

Hi,   I have data pulled from a form with a list of email addresses and names of people. I need to match them to data from a database to assign them to their account ID number. But people filling i...
  • jennratten's avatar
    jennratten
    5 years ago

    Great - I'm so glad we're making progress!  This line of the script determines which column number should be returned.

                else List.Last ( d{t} )  

     To get the column's position dynamically (instead of specifying the last column), add a step before add_dbAccount:

        column_dbPosition = List.PositionOf ( Table.ColumnNames ( dbData ), "Account ID" ),

    And then reference it instead, like this:

    else d{t}{column_dbPosition}  

     

    Here is the complete new script with the column number dynamically referenced.

    let
        // database data
        dbData = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("dY7BDoIwDIZfZdnZmKggcFs4ePCKN8JhaIUhY4bB+9uOgZBosnb9/q7/muf8aiy8a77DAjCnrakq65B0tnBjQGCUjvd3o1Gzo5XdpIhKS9V6/XA8BbzY5TyjByhk48bbyV/rfza/vlysb4pwyplWQz0Rm+tBaVGbYWXHMMIgStx4Cl0jtaLlsKTcg3xM5BpsFkovlMSb/bAjMDwFURw660sPNOeveR/CZbmnfAGQk4CVH54kjs68KD4=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"First Name" = _t, Nickname = _t, Surname = _t, #"Full Name" = _t, #"email 1" = _t, #"email 2" = _t, #"Account ID" = _t]),
        dbData_ChangeTypes = Table.TransformColumnTypes(dbData,{{"First Name", type text}, {"Nickname", type text}, {"Surname", type text}, {"Full Name", type text}, {"email 1", type text}, {"email 2", type text}, {"Account ID", Int64.Type}}), 
        // transform the table to a list of lists.
        dbData_List = Table.ToRows ( dbData_ChangeTypes ),
        // form data
        formData = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bY3BCoMwEER/ZclZ+g0hBwte7S14iHUaY90sxPT/K1sLFnqYy7zHjPdmEdg94yoxbpe7sGlMJyCnhRkab0Zku+eADplcQZiU1cR2lsohrQe/JaaeU52V/5vvX1vI54MIKRGLxWnmqh11kvGRHuEJFX60tmCi9ns4vAE=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Email = _t, Name = _t]),
        formData_ChangeTypes = Table.TransformColumnTypes(formData,{{"Email", type text}, {"Name", type text}}),
        // Extract delimited parts of the name, so they can be evaluated separately.
        // Take caution when using Text.Split as extra delimiters can throw off the results.
        // Ideally, the form fields would be similar in structure to the db to be matched, but that is not the case here.
        // Edit/add steps if additional delimited parts need to be accomodated, like name suffixes.
        formData_NamePart1 = Table.AddColumn(formData_ChangeTypes, "Name Part 1", each Text.BeforeDelimiter([Name], " "), type text),
        formData_NamePart2 = Table.AddColumn(formData_NamePart1, "Name Part 2", each Text.AfterDelimiter([Name], " ", {0, RelativePosition.FromEnd}), type text),
        // Add a list object to each row, containing each record's values for specific columns in the table.
        // If steps were added for additonal name parts, also add them to the list of fields (r) in this step.
        // Match the form data to the db data.
        column_dbPosition = List.PositionOf ( Table.ColumnNames ( dbData ), "Account ID" ),
        add_dbAccount = Table.AddColumn (
            formData_NamePart2, 
            "Account ID", 
            each 
            let                                              // Variables
                r = Record.ToList (                          // List of specific fields 
                    Record.SelectFields ( 
                        _, { "Email", "Name Part 1", "Name Part 2" } 
                    )
                ),                                           // Current record values as a list
                d = dbData_List,                             // db list of lists 
                t = List.PositionOf (                        // Position of match in the db
                    List.Transform (                         // --> list of 
                        d, each List.ContainsAll (           
                            _,                               // db lists
                            r,                               // containing form data elements
                            Comparer.FromCulture (           // Apply current culture 
                                Culture.Current, true        // with case insensitivity
                            )
                        )
                    ), true 
                ),
                i = if t = -1                                // = -1 if no matches were found
                then null                                     
                //else List.Last ( d{t} )     
                else d{t}{column_dbPosition}                 
            in 
                i,
            Int64.Type 
        ),
        remove_columns = Table.RemoveColumns (
            add_dbAccount,
            { "Name Part 1", "Name Part 2" }
        )
    in
        remove_columns

     

    Alternatively, you could use a function, like this: (I named the function fnGetDataFromExternalTable )

    let
        fn = ( formsTable as table, dbTable as table, dbColumnName as text ) =>
    let
        // database data
        dbData = dbTable,
        dbData_ChangeTypes = Table.TransformColumnTypes(dbData,{{"First Name", type text}, {"Nickname", type text}, {"Surname", type text}, {"Full Name", type text}, {"email 1", type text}, {"email 2", type text}, {"Account ID", Int64.Type}}), 
        // transform the table to a list of lists.
        dbData_List = Table.ToRows ( dbData_ChangeTypes ),
        // form data
        formData = formsTable,
        formData_ChangeTypes = Table.TransformColumnTypes(formData,{{"Email", type text}, {"Name", type text}}),
        // Extract delimited parts of the name, so they can be evaluated separately.
        // Take caution when using Text.Split as extra delimiters can throw off the results.
        // Ideally, the form fields would be similar in structure to the db to be matched, but that is not the case here.
        // Edit/add steps if additional delimited parts need to be accomodated, like name suffixes.
        formData_NamePart1 = Table.AddColumn(formData_ChangeTypes, "Name Part 1", each Text.BeforeDelimiter([Name], " "), type text),
        formData_NamePart2 = Table.AddColumn(formData_NamePart1, "Name Part 2", each Text.AfterDelimiter([Name], " ", {0, RelativePosition.FromEnd}), type text),
        // Add a list object to each row, containing each record's values for specific columns in the table.
        // If steps were added for additonal name parts, also add them to the list of fields (r) in this step.
        // Match the form data to the db data.
        column_dbPosition = List.PositionOf ( Table.ColumnNames ( dbData ), dbColumnName ),
        add_dbAccount = Table.AddColumn (
            formData_NamePart2, 
            dbColumnName, 
            each 
            let                                              // Variables
                r = Record.ToList (                          // List of specific fields 
                    Record.SelectFields ( 
                        _, { "Email", "Name Part 1", "Name Part 2" } 
                    )
                ),                                           // Current record values as a list
                d = dbData_List,                             // db list of lists 
                t = List.PositionOf (                        // Position of match in the db
                    List.Transform (                         // --> list of 
                        d, each List.ContainsAll (           
                            _,                               // db lists
                            r,                               // containing form data elements
                            Comparer.FromCulture (           // Apply current culture 
                                Culture.Current, true        // with case insensitivity
                            )
                        )
                    ), true 
                ),
                i = if t = -1                                // = -1 if no matches were found
                then null                                     
                //else List.Last ( d{t} )     
                else d{t}{column_dbPosition}                 
            in 
                i,
            Int64.Type 
        ),
        remove_columns = Table.RemoveColumns (
            add_dbAccount,
            { "Name Part 1", "Name Part 2" }
        )
    in
        remove_columns
    in 
        fn

    Invoke the function:

    fnGetDataFromExternalTable ( formsDataTable, dbDataTable, "Account ID" )