Forum Discussion
Add column with list.generate
- 5 years ago
Hello Anonymous
I would suggest you to do this with List.Accumulate. After you created the list that represents your index-column you have to integrate it your current table with Table.ToColumns and Table.FromColumns. Check out this example.
Give it a try and if you have performance issue we can check if we can enhance it a little bit
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlCK1YlWMgSTBkgkXCQWAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [A = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"A", Int64.Type}}), CreateNewIndexColumn = List.RemoveFirstN ( List.Accumulate ( #"Changed Type"[A], {1}, (state,current)=> state & {List.Last(state) + current} ) ) , AddIndexColumnToTable = Table.FromColumns ( Table.ToColumns(#"Changed Type" ) & {CreateNewIndexColumn}, Table.ColumnNames(#"Changed Type") & {"Index"} ) in AddIndexColumnToTableCopy paste this code to the advanced editor in a new blank query to see how the solution works.
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy - Anonymous5 years ago
ten millions rows worked in less than1'
let Source = List.Transform(List.Random(10000000), each Number.Round(_)), #"Converted to Table" = Table.FromList(Source, Splitter.SplitByNothing(), null, null, ExtraValues.Error), #"Grouped Rows" = Table.Group(#"Converted to Table", {"Column1"}, {{"Count", each Table.RowCount(_), Int64.Type}},GroupKind.Local), #"Added Index" = Table.AddIndexColumn(#"Grouped Rows", "Index", 1, 1, Int64.Type), #"Added Custom" = Table.AddColumn(#"Added Index", "idx", each List.Repeat({[Index]},[Count])), #"Expanded idx" = Table.ExpandListColumn(#"Added Custom", "idx") in #"Expanded idx"
Hello Anonymous
I would suggest you to do this with List.Accumulate. After you created the list that represents your index-column you have to integrate it your current table with Table.ToColumns and Table.FromColumns. Check out this example.
Give it a try and if you have performance issue we can check if we can enhance it a little bit
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlCK1YlWMgSTBkgkXCQWAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [A = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"A", Int64.Type}}),
CreateNewIndexColumn = List.RemoveFirstN
(
List.Accumulate
(
#"Changed Type"[A],
{1},
(state,current)=> state & {List.Last(state) + current}
)
)
,
AddIndexColumnToTable = Table.FromColumns
(
Table.ToColumns(#"Changed Type" ) & {CreateNewIndexColumn},
Table.ColumnNames(#"Changed Type") & {"Index"}
)
in
AddIndexColumnToTable
Copy paste this code to the advanced editor in a new blank query to see how the solution works.
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy
Thank you Jimmy801 it works perfect! do you have advice for how i can make it faster.
- Jimmy8015 years agoCommunity Champion
Hello Anonymous
how long it runs? How big is your dataset (columns/rows?)
BR
Jimmy
- Anonymous5 years agoNot applicable
Jimmy801 Daily data contains 1.5 million rows and 20 columns. It would take about 10 minutes to get data from the server before adding this column. This time it took more than 30 minutes. When I try to get data for 2 days or more, it gives this error: Expression.Error: The evaluation resulted in a stack capacity that is exceeded and can not continue.
- Anonymous5 years agoNot applicable
ten millions rows worked in less than1'
let Source = List.Transform(List.Random(10000000), each Number.Round(_)), #"Converted to Table" = Table.FromList(Source, Splitter.SplitByNothing(), null, null, ExtraValues.Error), #"Grouped Rows" = Table.Group(#"Converted to Table", {"Column1"}, {{"Count", each Table.RowCount(_), Int64.Type}},GroupKind.Local), #"Added Index" = Table.AddIndexColumn(#"Grouped Rows", "Index", 1, 1, Int64.Type), #"Added Custom" = Table.AddColumn(#"Added Index", "idx", each List.Repeat({[Index]},[Count])), #"Expanded idx" = Table.ExpandListColumn(#"Added Custom", "idx") in #"Expanded idx"