Forum Discussion
Data Translation
- 6 years ago
Here is one way to do it. To see how it works, just create a blank query, go to Advanced Editor, and replace the text there with the M code below.
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("hY47CoAwEESvElIr7m42iUmpWCripxLvfw0HLMQP2EzxhnnMtllWEmVfsi3sOCxIV4mvhIQMh+wVZJ27ie9NzBJBuLZ7cUlSAmvap0S/JYoiE5mxNxTQIF6yxyVMJLuEyZ8Rz857+wE=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [GRP = _t, USER_INPTS = _t, OBC_TS = _t, ACK_BY = _t, ACK_TS = _t, ERROR = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"GRP", type text}, {"USER_INPTS", type text}, {"OBC_TS", type text}, {"ACK_BY", type text}, {"ACK_TS", type text}, {"ERROR", Int64.Type}}), #"Grouped Rows" = Table.Group(#"Changed Type", {"GRP"}, {{"AllRows", each _, type table [GRP=text, USER_INPTS=text, OBC_TS=text, ACK_BY=text, ACK_TS=text, ERROR=number]}}), #"Added Custom" = Table.AddColumn(#"Grouped Rows", "WithIndex", each Table.AddIndexColumn([AllRows], "Index",1,1)), #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"AllRows"}), #"Expanded WithIndex" = Table.ExpandTableColumn(#"Removed Columns", "WithIndex", {"USER_INPTS", "OBC_TS", "ACK_BY", "ACK_TS", "ERROR", "Index"}, {"USER_INPTS", "OBC_TS", "ACK_BY", "ACK_TS", "ERROR", "Index"}), #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Expanded WithIndex", {"GRP", "Index"}, "Attribute", "Value"), #"Merged Columns" = Table.CombineColumns(Table.TransformColumnTypes(#"Unpivoted Other Columns", {{"Index", type text}}, "en-US"),{"Attribute", "Index"},Combiner.CombineTextByDelimiter("_", QuoteStyle.None),"Merged"), #"Pivoted Column" = Table.Pivot(#"Merged Columns", List.Distinct(#"Merged Columns"[Merged]), "Merged", "Value"), #"Changed Type1" = Table.TransformColumnTypes(#"Pivoted Column",{{"GRP", type text}, {"USER_INPTS_1", type text}, {"OBC_TS_1", type datetime}, {"ACK_BY_1", type text}, {"ACK_TS_1", type text}, {"ERROR_1", Int64.Type}, {"USER_INPTS_2", type text}, {"OBC_TS_2", type text}, {"ACK_BY_2", type text}, {"ACK_TS_2", type text}, {"ERROR_2", Int64.Type}}) in #"Changed Type1"If this works for you, please mark it as the solution. Kudos are appreciated too. Please let me know if not.
Regards,
Pat
It just got done running, it was quite slow, but manageable.
One thing that happened was on the final step, it resulted in 300+ columns. Is there a step maybe in the index where I can limit it to 6 errors per group? I think that would speed it up enough that would be acceptable.
And yes, 100% agree we need to adjust the data model, but don't have the support to do so at the moment.
I limited the the columns to just "ERROR" and this increased performance.
This solved my issue, thank you!