Forum Discussion
Matrix Excel data import to power bi in tablular format
- 5 years ago
Hello Mohammada
I suppose you are reading from a pivottable. This is always tricky and needs some specific techniques to find a result. In my proposed solution i first identify the planned columns and the actual columns. Then the tables are separated and applied some steps liks removing tops rows, promote headers and and unpivot inclusive filter. Then both are again combines and pivoted
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("pVZfb9owEP8qFk+d5lLHiQM8Ulq0SV2HSqc+VH1wE9NEBDtynGp8m32WfbLZToCM2INtknPOkbvf/fHdmefnweJuen9/ezOAg/6azh6/Te+Wxx9e4LPZERkGaIgRRoaJu8yoy4w7zH79izKa7JnWhS+Ca/pYM02fWGres9pQoWhR7Y35xOYyt0jXIi+YBCnbUJ6CCyU4Z9UHZ0qua77WopUSyRpQpV+oVECsQEq34OI9r2pagCRjybqn37EU/PyhfwkxcdLfLAUm8N7TwcLnIuFTSOG5SOEppOj8+KJTWG4MBxLxItly0FwQWy3n1iwrCV47h1z5UTMp6resrBUoRKWAEqDmZUF18ZgSi3CfuFdbzP31hX4f7pxJaEmTXG2Nx8hJDMyiMa+VpCn5AMbEuwWQkL90aJqo2uZHcrDK32rJvNnxRnts6Kbpug1TYCXFxil4QsbALG1HXoiS8Zy/gVeqM5Ew04QYTkzBwDgwvRRBPLLnAcfjcA9xSrsf32euD75yeuv+ZEv6YXY1Xyw0SwhyUgxH7aEcZN0J/iQkr9gWLJVkTA1NSCPkpBiGLWZfx4PNeCq4+R5EyElHqB3nB1E31P3d0xSkuWSJ7pC8LHV6jTo5gMTNu6VhU5t6ufU87s5m4IkqJlNaMHAFDK+HgtI6KyFNM+ImuT2qe6ANowvxh8RosZ5TY9InEe4AnxnGrQ7ZFALqk8ZbU6gZ5esKLB/mcy/OV5XpgaGyXKag1NeTvpiWppRj1Cfh7hxnQidMbiu1LfxNHYwvUXCJIpMgfPRMWpwi53lCC38OrXuVLdijZ/cX4SDhRthNuX2j6QqPg2YPmh3veLvrWTch0aA7w/a6/zm+2smhr3+VbxgoGbU3P2k9IHCCTXZiSEIzbkYQR+E5gIm+U45GkW+EEUgmvVHm0PeNskZjI97ZhnEFPl5dGketqdgG0ZhtfolQ1LVzpOVI5ssv", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t, Column3 = _t, Column4 = _t, Column5 = _t, Column6 = _t, Column7 = _t, Column8 = _t, Column9 = _t, Column10 = _t, Column11 = _t, Column12 = _t, Column13 = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}, {"Column8", type text}, {"Column9", type text}, {"Column10", type text}, {"Column11", type text}, {"Column12", type text}, {"Column13", type text}}), IdentifyColumnWithCellActual = List.Select ( List.Transform ( Table.ColumnNames(#"Changed Type"), (trans)=> if List.Count(List.Select(Table.Column(#"Changed Type", trans),each _ = "ACTUALS" ))>0 then trans else null ), each _ <> null ){0}, IdentifyPlannedColumns = List.FirstN ( Table.ColumnNames(#"Changed Type"), each _ <> IdentifyColumnWithCellActual ), IdentifyActualColumns = List.Difference ( Table.ColumnNames(#"Changed Type"), IdentifyPlannedColumns ), CreatePlannedTable = let SelColumn = Table.SelectColumns(#"Changed Type", IdentifyPlannedColumns), RemoveTop1 = Table.Skip(SelColumn,1), PromoteHeaders = Table.PromoteHeaders(RemoveTop1, [PromoteAllScalars=true]), RemoveTop2 = Table.Skip(PromoteHeaders,1), UnpivotOtherThenFirst = Table.UnpivotOtherColumns(RemoveTop2, {""}, "Attribute", "Value"), TableSelectRows = Table.SelectRows ( UnpivotOtherThenFirst, each Value.Is(try Date.From(_[Attribute]) otherwise "", type date) and _[Value]<>"" ), AddPlan = Table.AddColumn(TableSelectRows, "Type", each "Planned") in AddPlan, CreateActualTable = let SelColumn = Table.SelectColumns(#"Changed Type", IdentifyActualColumns), RemoveTop1 = Table.Skip(SelColumn,1), PromoteHeaders = Table.PromoteHeaders(RemoveTop1, [PromoteAllScalars=true]), RemoveTop2 = Table.Skip(PromoteHeaders,1), UnpivotOtherThenFirst = Table.UnpivotOtherColumns(RemoveTop2, {""}, "Attribute", "Value"), TableSelectRows = Table.SelectRows ( UnpivotOtherThenFirst, each Value.Is(try Date.From(_[Attribute]) otherwise "", type date) and _[Value]<>"" ), AddActual = Table.AddColumn(TableSelectRows, "Type", each "Actual") in AddActual, CombineBoth =Table.Combine({CreatePlannedTable, CreateActualTable}), #"Changed Type1" = Table.TransformColumnTypes(CombineBoth,{{"Value", type number}}), #"Pivoted Column" = Table.Pivot(#"Changed Type1", List.Distinct(#"Changed Type1"[#""]), "", "Value", List.Sum) in #"Pivoted Column"Copy paste this code to the advanced editor in a new blank query to see how the solution works.
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy - 5 years ago
Hello Mohammada
if the code worked properly I would ask you to mark it as solution. About your request of Mail, you could use Exchange.Contents to read your Mailbox and in case apply filters according to your logic (sender, time, subject) and then read the according appendix
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy
Hi Jimmy,
Thanks, I have accepted it as a solution and tried to use exchange and attachments. However, I receive the following error with the same code I have used.
Expression.Error: We cannot convert a value of type Table to type List.
Details:
Value=[Table]
Type=[Type].
Please see the below code.
let
Source = Table.FromRows(#"Mail (2)",let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t, Column3 = _t, Column4 = _t, Column5 = _t, Column6 = _t, Column7 = _t, Column8 = _t, Column9 = _t, Column10 = _t, Column11 = _t, Column12 = _t, Column13 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}, {"Column8", type text}, {"Column9", type text}, {"Column10", type text}, {"Column11", type text}, {"Column12", type text}, {"Column13", type text}}),
IdentifyColumnWithCellActual = List.Select
(
List.Transform
(
Table.ColumnNames(#"Changed Type"),
(trans)=> if List.Count(List.Select(Table.Column(#"Changed Type", trans),each _ = "ACTUALS" ))>0 then trans else null
),
each _ <> null
){0},
IdentifyPlannedColumns = List.FirstN
(
Table.ColumnNames(#"Changed Type"),
each _ <> IdentifyColumnWithCellActual
),
IdentifyActualColumns = List.Difference
(
Table.ColumnNames(#"Changed Type"),
IdentifyPlannedColumns
),
CreatePlannedTable =
let
SelColumn = Table.SelectColumns(#"Changed Type", IdentifyPlannedColumns),
RemoveTop1 = Table.Skip(SelColumn,1),
PromoteHeaders = Table.PromoteHeaders(RemoveTop1, [PromoteAllScalars=true]),
RemoveTop2 = Table.Skip(PromoteHeaders,1),
UnpivotOtherThenFirst = Table.UnpivotOtherColumns(RemoveTop2, {""}, "Attribute", "Value"),
TableSelectRows = Table.SelectRows
(
UnpivotOtherThenFirst,
each Value.Is(try Date.From(_[Attribute]) otherwise "", type date) and _[Value]<>""
),
AddPlan = Table.AddColumn(TableSelectRows, "Type", each "Planned")
in
AddPlan,
CreateActualTable =
let
SelColumn = Table.SelectColumns(#"Changed Type", IdentifyActualColumns),
RemoveTop1 = Table.Skip(SelColumn,1),
PromoteHeaders = Table.PromoteHeaders(RemoveTop1, [PromoteAllScalars=true]),
RemoveTop2 = Table.Skip(PromoteHeaders,1),
UnpivotOtherThenFirst = Table.UnpivotOtherColumns(RemoveTop2, {""}, "Attribute", "Value"),
TableSelectRows = Table.SelectRows
(
UnpivotOtherThenFirst,
each Value.Is(try Date.From(_[Attribute]) otherwise "", type date) and _[Value]<>""
),
AddActual = Table.AddColumn(TableSelectRows, "Type", each "Actual")
in
AddActual,
CombineBoth =Table.Combine({CreatePlannedTable, CreateActualTable}),
#"Changed Type1" = Table.TransformColumnTypes(CombineBoth,{{"Value", type number}}),
#"Pivoted Column" = Table.Pivot(#"Changed Type1", List.Distinct(#"Changed Type1"[#""]), "", "Value", List.Sum)
in
#"Pivoted Column"
Hello Mohammada
if the query #"Mail(2)" is delivering a table from Excel-sheet in your mail-account change the frist step like this
Source = #"Mail(2)",
Be aware that the table deliverd to my code has to have no real column headers, only anonymous "column1" etc.
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy
- Mohammada5 years agoRegular Visitor
Hi Jimmy,
I have tried other way but it didnt work. It gave me the following error.The columns I have in my table have column1,column2 so on not the standard column headers.
Please click below for power bi table:
Please click here for the original table: https://londonenergy-my.sharepoint.com/:x:/g/personal/mohammada_londonenergyltd_com/EaC5RyxmwwNGurbVV6xEZYUBHnpXakQl7rwpoNlIaxsx8A?e=BxzuLp
- Jimmy8015 years ago
Community Champion
Hello Mohammada
First create a query that produces the output you posted in your first post (anonymous columns...). Call this query "MyTable".
Then create a new blank query and copy paste this code
let Source = MyTable IdentifyColumnWithCellActual = List.Select ( List.Transform ( Table.ColumnNames(Source), (trans)=> if List.Count(List.Select(Table.Column(Source, trans),each _ = "ACTUALS" ))>0 then trans else null ), each _ <> null ){0}, IdentifyPlannedColumns = List.FirstN ( Table.ColumnNames(Source), each _ <> IdentifyColumnWithCellActual ), IdentifyActualColumns = List.Difference ( Table.ColumnNames(Source), IdentifyPlannedColumns ), CreatePlannedTable = let SelColumn = Table.SelectColumns(Source, IdentifyPlannedColumns), RemoveTop1 = Table.Skip(SelColumn,1), PromoteHeaders = Table.PromoteHeaders(RemoveTop1, [PromoteAllScalars=true]), RemoveTop2 = Table.Skip(PromoteHeaders,1), UnpivotOtherThenFirst = Table.UnpivotOtherColumns(RemoveTop2, {""}, "Attribute", "Value"), TableSelectRows = Table.SelectRows ( UnpivotOtherThenFirst, each Value.Is(try Date.From(_[Attribute]) otherwise "", type date) and _[Value]<>"" ), AddPlan = Table.AddColumn(TableSelectRows, "Type", each "Planned") in AddPlan, CreateActualTable = let SelColumn = Table.SelectColumns(Source, IdentifyActualColumns), RemoveTop1 = Table.Skip(SelColumn,1), PromoteHeaders = Table.PromoteHeaders(RemoveTop1, [PromoteAllScalars=true]), RemoveTop2 = Table.Skip(PromoteHeaders,1), UnpivotOtherThenFirst = Table.UnpivotOtherColumns(RemoveTop2, {""}, "Attribute", "Value"), TableSelectRows = Table.SelectRows ( UnpivotOtherThenFirst, each Value.Is(try Date.From(_[Attribute]) otherwise "", type date) and _[Value]<>"" ), AddActual = Table.AddColumn(TableSelectRows, "Type", each "Actual") in AddActual, CombineBoth =Table.Combine({CreatePlannedTable, CreateActualTable}), #"Changed Type1" = Table.TransformColumnTypes(CombineBoth,{{"Value", type number}}), #"Pivoted Column" = Table.Pivot(#"Changed Type1", List.Distinct(#"Changed Type1"[#""]), "", "Value", List.Sum) in #"Pivoted Column"If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy- Mohammada5 years agoRegular Visitor
Hi Jimmy,
Please see the below error:
Many Thanks,
Mohammad