Forum Discussion
ctho222
1 year agoFrequent Visitor
Separating values from columns where there are multiple attribute values in a single column
SO! I was given a log document that staff in a different medical unit input various metrics they want to observe[see below example of the LOG]. I was told to smoothen the user experience of inputting...
- 1 year ago
If I can assume that the Log Headers are the same from log to log then you code do something like this...
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("nVJdb4IwFP0rjXt1fBk18lbaOtikkK4YjfGBaDdIiBDA/f65vfQya4wjhNx7OLfn3tuz240olmy0H+9Gvx/X9mzP8aYaQToUGUdPOt1snjc/j5EbiyEVsCiIY1V8le25M55xN+Q4Br3Lc3tS7Ri91gr0rD4sFNZdY2ksyJu+7HoUqrzqi0cEaRLoxHHty+suZos/cjHVwFt5+jQvky0tlPFIaohRIzPiy8Q3/pHblEGeZGKJSbSK5FbDWHKwb0KSrbkfwJKZ4Ew8sJcUy4hxMErKaISliAjY3VrYaVnVvYbe40iGxgOTvlDt/0whGY6BKgbjCrKmiORVBQzv+Y6DYAW87rJr8v5QALo/uckW6nBlNRK8BPcaZtQOVVU2+RF4a+a7jpl9autzr4bUye2GoBnTNj8WZ+hBcuVBcO24bS3042ggNvcn7jAF2lQ1hoKpM0ynVwoHWLDw3TkwLJNhAkbAcZCtMCcX3++/AQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]), header_list = {"DATE", "DX", "EF. HOSP.", "REF. MD", "ATN", "DVR/PILOT", "RCVD CALL", "ED/HELIPAD", "ARR. REF.", "RUN #", "NAME", "DOB", "REF. UNIT", "RN", "OTHER", "DISPATCH", "ENROUTE", "DEP. REF.", "MR #", "INFO:", "REC. HOSP.", "REC. MD", "TYPE", "PATIENT", "TEAM", "REC. UNIT", "ARR. REC.", "METHOD", "REF. HOSP."}, remove_blank_rows = Table.SelectRows(Source, each ([Column1] <> "" and [Column1] <> " ")), add_isHeader_column = Table.AddColumn(remove_blank_rows, "isHeader", each if List.Contains(header_list, [Column1], Comparer.OrdinalIgnoreCase) then [Column1] else null, type text), fill_down_isHeader = Table.FillDown(add_isHeader_column,{"isHeader"}), group_by_isHeader = Table.Group(fill_down_isHeader, {"isHeader"}, {{"All Rows", each _, type table [Column1=nullable text, isHeader=text]}}), select_nested_table_values = Table.TransformColumns(group_by_isHeader, {{"All Rows", each Table.SelectColumns(Table.SelectRows(_, each [Column1] <> [isHeader]), {"Column1"})}}), add_nested_index = Table.TransformColumns(select_nested_table_values, {{"All Rows", each Table.AddIndexColumn(_, "Index", 1, 1)}}), expand_nested_table = Table.ExpandTableColumn(add_nested_index, "All Rows", {"Column1", "Index"}), pivot_table = Table.Pivot(expand_nested_table, List.Distinct(expand_nested_table[isHeader]), "isHeader", "Column1"), remove_null_index = Table.SelectRows(pivot_table, each ([Index] <> null)), remove_index_column = Table.RemoveColumns(remove_null_index,{"Index"}), set_data_types = Table.TransformColumnTypes(remove_index_column,{{"DATE", type date}, {"RUN #", type text}, {"MR #", type text}, {"DX", type text}, {"NAME", type text}, {"Ref. Hosp.", type text}, {"DOB", type date}, {"Ref. MD", type text}, {"REF. UNIT", type text}, {"INFO:", type any}, {"TYPE", type text}, {"ATN", type text}, {"RN", type text}, {"PATIENT", type text}, {"DVR/Pilot", type text}, {"Other", type any}, {"TEAM", type text}, {"RCVD Call", type time}, {"Dispatch", type time}, {"Rec. Hosp.", type text}, {"ED/Helipad", type time}, {"Enroute", type time}, {"Rec. MD", type text}, {"REC. UNIT", type text}, {"Arr. Ref.", type time}, {"Dep. Ref.", type time}, {"Arr. Rec.", type time}, {"METHOD", type text}}) in set_data_typesThe basic idea is to set a list of the headers, compare the existing column to the header list, group by header names, select the coresponding values, pivot the table to end up with...
Hope this helps.
jgeddes
1 year agoSuper User
If I can assume that the Log Headers are the same from log to log then you code do something like this...
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("nVJdb4IwFP0rjXt1fBk18lbaOtikkK4YjfGBaDdIiBDA/f65vfQya4wjhNx7OLfn3tuz240olmy0H+9Gvx/X9mzP8aYaQToUGUdPOt1snjc/j5EbiyEVsCiIY1V8le25M55xN+Q4Br3Lc3tS7Ri91gr0rD4sFNZdY2ksyJu+7HoUqrzqi0cEaRLoxHHty+suZos/cjHVwFt5+jQvky0tlPFIaohRIzPiy8Q3/pHblEGeZGKJSbSK5FbDWHKwb0KSrbkfwJKZ4Ew8sJcUy4hxMErKaISliAjY3VrYaVnVvYbe40iGxgOTvlDt/0whGY6BKgbjCrKmiORVBQzv+Y6DYAW87rJr8v5QALo/uckW6nBlNRK8BPcaZtQOVVU2+RF4a+a7jpl9autzr4bUye2GoBnTNj8WZ+hBcuVBcO24bS3042ggNvcn7jAF2lQ1hoKpM0ynVwoHWLDw3TkwLJNhAkbAcZCtMCcX3++/AQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
header_list = {"DATE", "DX", "EF. HOSP.", "REF. MD", "ATN", "DVR/PILOT", "RCVD CALL", "ED/HELIPAD", "ARR. REF.", "RUN #", "NAME", "DOB", "REF. UNIT", "RN", "OTHER", "DISPATCH", "ENROUTE", "DEP. REF.", "MR #", "INFO:", "REC. HOSP.", "REC. MD", "TYPE", "PATIENT", "TEAM", "REC. UNIT", "ARR. REC.", "METHOD", "REF. HOSP."},
remove_blank_rows = Table.SelectRows(Source, each ([Column1] <> "" and [Column1] <> " ")),
add_isHeader_column = Table.AddColumn(remove_blank_rows, "isHeader", each if List.Contains(header_list, [Column1], Comparer.OrdinalIgnoreCase) then [Column1] else null, type text),
fill_down_isHeader = Table.FillDown(add_isHeader_column,{"isHeader"}),
group_by_isHeader = Table.Group(fill_down_isHeader, {"isHeader"}, {{"All Rows", each _, type table [Column1=nullable text, isHeader=text]}}),
select_nested_table_values = Table.TransformColumns(group_by_isHeader, {{"All Rows", each Table.SelectColumns(Table.SelectRows(_, each [Column1] <> [isHeader]), {"Column1"})}}),
add_nested_index = Table.TransformColumns(select_nested_table_values, {{"All Rows", each Table.AddIndexColumn(_, "Index", 1, 1)}}),
expand_nested_table = Table.ExpandTableColumn(add_nested_index, "All Rows", {"Column1", "Index"}),
pivot_table = Table.Pivot(expand_nested_table, List.Distinct(expand_nested_table[isHeader]), "isHeader", "Column1"),
remove_null_index = Table.SelectRows(pivot_table, each ([Index] <> null)),
remove_index_column = Table.RemoveColumns(remove_null_index,{"Index"}),
set_data_types = Table.TransformColumnTypes(remove_index_column,{{"DATE", type date}, {"RUN #", type text}, {"MR #", type text}, {"DX", type text}, {"NAME", type text}, {"Ref. Hosp.", type text}, {"DOB", type date}, {"Ref. MD", type text}, {"REF. UNIT", type text}, {"INFO:", type any}, {"TYPE", type text}, {"ATN", type text}, {"RN", type text}, {"PATIENT", type text}, {"DVR/Pilot", type text}, {"Other", type any}, {"TEAM", type text}, {"RCVD Call", type time}, {"Dispatch", type time}, {"Rec. Hosp.", type text}, {"ED/Helipad", type time}, {"Enroute", type time}, {"Rec. MD", type text}, {"REC. UNIT", type text}, {"Arr. Ref.", type time}, {"Dep. Ref.", type time}, {"Arr. Rec.", type time}, {"METHOD", type text}})
in
set_data_types
The basic idea is to set a list of the headers, compare the existing column to the header list, group by header names, select the coresponding values, pivot the table to end up with...
Hope this helps.
- ctho2221 year agoFrequent Visitor
Works for me and now the spreadsheet doesn't chug when it runs thank ya kindly!