Forum Discussion
Merging / appending to keep all
- 1 year ago
Hi j1s, I recommend Table.Join function for this purpose:
Output
let Table1 = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W8kvMTVUwVNJRgiNDpVgdqLgRTARMGSEkjJE1IIRNoIoNMUwyxWWSGaoWhIQ5disskNQboshYYtdgaIBitVJsLAA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Full Name" = _t, #"2024-11-25" = _t, #"2024-12-02" = _t, #"2024-12-09" = _t, #"2024-12-16" = _t]), Table2 = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W8kvMTVUwVNJRAiI4FasDlTCCiBhDJY0QMsYQGQwdJkiCQAoubooQRJUwQ7UCyShzhAyq3RaoZiFpscSlxdAApx5DCwOcuiwMMbTFAgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Full Name" = _t, #"2025-01-01" = _t, #"2025-01-08" = _t, #"2025-01-15" = _t, #"2025-01-22" = _t]), T2_RenamedColumn = Table.RenameColumns(Table2,{{"Full Name", "Full Name2"}}), Merged = Table.Join(Table1, "Full Name", T2_RenamedColumn, "Full Name2", JoinKind.FullOuter), ReplacedFullName = Table.ReplaceValue(Merged, each [Full Name] is null, each [Full Name2], (x,y,z)=> if y then z else x, {"Full Name"} ), RemovedColumns = Table.RemoveColumns(ReplacedFullName,{"Full Name2"}) in RemovedColumns - 1 year ago
Sorting the columns is not that easy, because your column names are text and we need a date sort.
But this works:
let #"Unpivoted Table1" = Table.UnpivotOtherColumns(Table1, {"Full Name"}, "Attribute", "Value"), #"Unpivoted Table2" = Table.UnpivotOtherColumns(Table2, {"Full Name"}, "Attribute", "Value"), #"Unpivoted Table3" = Table.UnpivotOtherColumns(Table3, {"Full Name"}, "Attribute", "Value"), #"Unpivoted ALL"= Table.Combine({#"Unpivoted Table1",#"Unpivoted Table2",#"Unpivoted Table3"}), #"Added Custom" = Table.AddColumn(#"Unpivoted ALL", "Dates", each [Attribute]), #"Changed Type" = Table.TransformColumnTypes(#"Added Custom",{{"Dates", type date}}), #"Grouped Rows" = Table.Group(#"Changed Type", {"Attribute", "Dates"}, {{"Count", each Table.RowCount(_), Int64.Type}}), #"Sorted Rows" = Table.Sort(#"Grouped Rows",{{"Dates", Order.Ascending}}), #"Pivoted Column" = Table.Pivot(#"Unpivoted ALL", List.Distinct(#"Unpivoted ALL"[Attribute]), "Attribute", "Value", List.Sum), #"Reordered Columns" = Table.ReorderColumns(#"Pivoted Column",#"Sorted Rows"[Attribute]) in #"Reordered Columns"Producing this:
About the missing dates: I suspect this will not be a problem in real life.
But if it is:
- create a table with all possible dates, for exapmple by querying the dat range in the orignal dataset
- Transpose the table
- Use Forst Row as Headers
- And if needed, remove all rows with a Table.SelectRows(#"Promote Headers", each false)
- and then do a Table.Union of this table with the pivoted table
I doubt this will be worth the trouble ....
Did I answer your question? Then please (also) mark my post as a solution and make it easier to find for others having a similar problem.
Remember: You can mark multiple answers as a solution...
If I helped you, please click on the Thumbs Up to give Kudos.Kees Stolker
A big fan of Power Query and Excel
- j1s1 year agoHelper I
Thanks for proposing this solution.
Since my sources references are slightly different, I took your code:
let Table1 = Excel.CurrentWorkbook(){[Name = "Table1"]}[Content], Table2 = Excel.CurrentWorkbook(){[Name = "Table2"]}[Content], ColNames = List.RemoveFirstN(Table.ColumnNames(Table1),1) & List.RemoveFirstN(Table.ColumnNames(Table2),1), TranspTable1 = Table.PromoteHeaders(Table.FromRows(Table.ToColumns(Excel.CurrentWorkbook(){[Name = "Table1"]}[Content]))), TranspTable2 = Table.PromoteHeaders(Table.FromRows(Table.ToColumns(Excel.CurrentWorkbook(){[Name = "Table2"]}[Content]))), Table = Table.FromRows(Table.ToColumns(TranspTable1 & TranspTable2),ColNames), #"Added Index" = Table.TransformColumns(Table.AddIndexColumn(Table, "Name", 0, 1, Int64.Type),{"Name", each List.Distinct(Table1[Column1] & Table2[Column1]){_}}), #"Reordered Columns" = Table.ReorderColumns(#"Added Index",{"Name", "01-01-2025", "01-02-2025", "01-03-2025", "01-04-2025", "01-05-2025", "01-06-2025", "01-07-2025", "01-08-2025", "01-09-2025"}) in #"Reordered Columns"...and tried to modify it like this:
let Source = Excel.CurrentWorkbook(){[Name="BaseData"]}[Content], Table1 = Source, Table2 = #"Latest Update", //refers to an additional query that gets the latest data from another file and puts it in the same format as "Table1" ColNames = List.RemoveFirstN(Table.ColumnNames(Table1),1) & List.RemoveFirstN(Table.ColumnNames(Table2),1), // modified these 2 lines: TranspTable1 = Table.PromoteHeaders(Table.FromRows(Table.ToColumns(Table1))), TranspTable2 = Table.PromoteHeaders(Table.FromRows(Table.ToColumns(Table2))), // from here it's the same as @SundarRaj orginal, but there's an error from the next step Table = Table.FromRows(Table.ToColumns(TranspTable1 & TranspTable2),ColNames), #"Added Index" = Table.TransformColumns(Table.AddIndexColumn(Table, "Name", 0, 1, Int64.Type),{"Name", each List.Distinct(Table1[Column1] & Table2[Column1]){_}}), #"Reordered Columns" = Table.ReorderColumns(#"Added Index",{"Name", "01-01-2025", "01-02-2025", "01-03-2025", "01-04-2025", "01-05-2025", "01-06-2025", "01-07-2025", "01-08-2025", "01-09-2025"}) in #"Reordered Columns"The error I get at
Table = Table.FromRows(Table.ToColumns(TranspTable1 & TranspTable2),ColNames)is:
Expression.Error: The field '01/01/2025' already exists in the record. Details: Name=01/01/2025 Value=- PwerQueryKees1 year agoSuper User
What the error says is that
ColNames = List.RemoveFirstN(Table.ColumnNames(Table1),1) & List.RemoveFirstN(Table.ColumnNames(Table2),1),Produces duplicate column names and the Table.FromRows() does not like that. Can't see the actual data you are using, so I am not sure...