Forum Discussion
H_George
3 years agoFrequent Visitor
Get all rows from a table based on multiple values on my current table
Hi everyone, I have 2 large dataset where there are no unique values. What I would like to do is import every value from my second table to the first table where the Material IDs match, and in one ...
- 3 years ago
Hi, try this. Should work faster.
let // table 01 data tbl1 = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMjY2VtJRMjDSNzLQNwIxQzJzU+MNlWJ14HLGeORM8OszBcrFlKDKmpiYAAUM9Q2BWsE6S0zB4qampkCOEUI8zUQpNhYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [material_id = _t, dates_01 = _t, #"Capacity needed_OLD" = _t]), type_01 = Table.TransformColumnTypes(tbl1,{{"dates_01", type date}}), // group table 01 by material_id and sort each table by date g1 = Table.Group(type_01, {"material_id"}, {{"tbl1", each Table.Sort(_, {"dates_01", Order.Descending})}}), // table 02 data tbl2 = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMjY2VtJRMjDWNzTVNwIxQzJzU+ONlGJ14HImqHLGYDkjIyOggKG+ob6RAViuBCRnbm4OljUxMcEia2lpCZY1NTWFyEKNBUuCBGNjAQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [material_id = _t, dates_02 = _t, #"Improved Time" = _t]), type_02 = Table.TransformColumnTypes(tbl2,{{"dates_02", type date}}), // group table by material_id and make a record (dict) with material_id as field names g2 = Table.Group(type_02, {"material_id"}, {{"Value", each List.Zip({[dates_02], [Improved Time]})}}), dict = Record.FromTable(Table.RenameColumns(g2,{{"material_id", "Name"}})), // this is join replacement. Should work faster! tbl = Table.AddColumn(g1, "tbl2", each Record.FieldOrDefault(dict, [material_id], null)), // this function iterates table "old" (resides in tbl1 column) and grabs date from tbl2 fx_new_capacity = (old as table, new as list) => let old_count = Table.RowCount(old), dates = List.Buffer( List.Sort( old[dates_01], Order.Descending ) ), updates = List.Buffer( List.Sort( new, {each _{0}, Order.Descending} ) ), new_column = List.Generate( () => [ i = 0, lst = updates, new_col = {}], (x) => x[i] <= old_count and not List.IsEmpty( x[lst] ), (x) => let a = List.Skip ( x[lst], each _{0} >= dates{x[i]} ), b = List.First( a ){1} in [ i = x[i] + 1, lst = a, new_col = x[new_col] & {b} ] ), out = Table.FromColumns( Table.ToColumns(old) & {List.Last(new_column)[new_col]}, Table.ColumnNames(old) & {"Capacity_neededNEW"} ) in out, // here we iterate table tbl and call our function, then combine our results into final table new_cap = Table.Combine(Table.TransformRows( tbl, each if [tbl2] = null then [tbl1] else fx_new_capacity( [tbl1], [tbl2] ))) in new_cap
AlienSx
3 years agoSuper User
Hi, try this. Should work faster.
let
// table 01 data
tbl1 = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMjY2VtJRMjDSNzLQNwIxQzJzU+MNlWJ14HLGeORM8OszBcrFlKDKmpiYAAUM9Q2BWsE6S0zB4qampkCOEUI8zUQpNhYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [material_id = _t, dates_01 = _t, #"Capacity needed_OLD" = _t]),
type_01 = Table.TransformColumnTypes(tbl1,{{"dates_01", type date}}),
// group table 01 by material_id and sort each table by date
g1 = Table.Group(type_01, {"material_id"}, {{"tbl1", each Table.Sort(_, {"dates_01", Order.Descending})}}),
// table 02 data
tbl2 = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMjY2VtJRMjDWNzTVNwIxQzJzU+ONlGJ14HImqHLGYDkjIyOggKG+ob6RAViuBCRnbm4OljUxMcEia2lpCZY1NTWFyEKNBUuCBGNjAQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [material_id = _t, dates_02 = _t, #"Improved Time" = _t]),
type_02 = Table.TransformColumnTypes(tbl2,{{"dates_02", type date}}),
// group table by material_id and make a record (dict) with material_id as field names
g2 = Table.Group(type_02, {"material_id"}, {{"Value", each List.Zip({[dates_02], [Improved Time]})}}),
dict = Record.FromTable(Table.RenameColumns(g2,{{"material_id", "Name"}})),
// this is join replacement. Should work faster!
tbl = Table.AddColumn(g1, "tbl2", each Record.FieldOrDefault(dict, [material_id], null)),
// this function iterates table "old" (resides in tbl1 column) and grabs date from tbl2
fx_new_capacity = (old as table, new as list) =>
let
old_count = Table.RowCount(old),
dates = List.Buffer( List.Sort( old[dates_01], Order.Descending ) ),
updates = List.Buffer( List.Sort( new, {each _{0}, Order.Descending} ) ),
new_column =
List.Generate(
() => [ i = 0, lst = updates, new_col = {}],
(x) => x[i] <= old_count and not List.IsEmpty( x[lst] ),
(x) =>
let
a = List.Skip ( x[lst], each _{0} >= dates{x[i]} ),
b = List.First( a ){1}
in [ i = x[i] + 1, lst = a, new_col = x[new_col] & {b} ]
),
out = Table.FromColumns( Table.ToColumns(old) & {List.Last(new_column)[new_col]}, Table.ColumnNames(old) & {"Capacity_neededNEW"} )
in out,
// here we iterate table tbl and call our function, then combine our results into final table
new_cap = Table.Combine(Table.TransformRows( tbl, each if [tbl2] = null then [tbl1] else fx_new_capacity( [tbl1], [tbl2] )))
in
new_capH_George
3 years agoFrequent Visitor
Thank you very much, this works perfectly.