Forum Discussion
Matrix Excel data import to power bi in tablular format
Hi,
I am new to BI anyone guidelines would be really helpful. I have a daily report in matrix format of Excel which we would like to visualize in power bi. the user adds the data in excel for before day stock values and we want to present it to managers every day with visuals. As it is in a matrix format, if I import the excel in BI it will not export the right columns. Is there a way to automate the matrix data to tabular form and export it to Power bi?
| PLANNED |
|
|
|
|
|
| ACTUALS |
|
|
|
|
|
|
| 05/10/2020 | 06/10/2020 | 07/10/2020 | 08/10/2020 |
|
|
| 05/10/2020 | 06/10/2020 | 07/10/2020 | 08/10/2020 | 09/10/2020 |
|
| Mon | Tue | Wed | Thu | Totals |
|
| Mon | Tue | Wed | Thu | Fri |
| Boiler demand (tonnes) |
|
|
|
|
|
| Bunker stock at start of day (visual check) |
|
|
|
|
|
| Boiler 1 | 325 | 325 | 325 | 325 |
|
| Bunker 1 | 0 | 0 | 0 | 0 | 0 |
| Boiler 2 | 325 | 325 | 325 | 325 |
|
| Bunker 2 | 0 | 0 | 0 | 0 | 0 |
| Boiler 3 | 325 | 325 | 325 | 325 |
|
| Bunker 3 | 0 | 0 | 0 | 0 | 0 |
| Boiler 4 | 325 | 325 | 325 | 325 |
|
| Bunker 4 | 0 | 0 | 0 | 0 | 0 |
| Boiler 5 | 325 | 325 | 325 | 325 |
|
| Bunker 5 | 0 | 0 | 0 | 0 | 0 |
| Total | 1625 | 1625 | 1625 | 1625 |
|
| Total bunker stocks | 0 | 0 | 0 | 0 | 0 |
| Throughput lost to unplanned | 42 | 42 | 42 | 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Max. bunker capacity | 10,000 | 10,000 | 10,000 | 10,000 | 10,000 |
| Planned burn | 1,650 | 1,650 | 1,650 | 1,650 | 11,550 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Actual burn figure | 0 | 0 | 0 | 0 | 0 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Demand met from |
|
|
|
|
|
| Demand met from |
|
|
|
|
|
| Stock (opening balance) | 2,900 | 3,611 | 4,272 | 4,883 |
|
| Stock (opening balance) | 2,900 | 0 | 0 | 0 | 0 |
| Inputs |
|
|
|
|
|
| Inputs |
|
|
|
|
|
| BRC/FPP | 550 | 550 | 550 | 550 | 2,750 |
| BRC/FPP | 0 | 0 | 0 | 0 | 0 |
| Hornsey Street. | 470 | 470 | 470 | 470 | 2,350 |
| Hornsey Street. | 0 | 0 | 0 | 0 | 0 |
| Hendon. | 140 | 140 | 140 | 140 | 700 |
| Hendon. | 0 | 0 | 0 | 0 | 0 |
| NLWA direct tipping | 750 | 700 | 650 | 750 | 3,650 |
| NLWA direct tipping | 0 | 0 | 0 | 0 | 0 |
| HCC Waterdale / HCC Buntingford | 250 | 250 | 250 | 250 | 1,500 |
| HCC Waterdale. | 0 | 0 | 0 | 0 | 0 |
| HCC direct tipping | 85 | 85 | 85 | 85 | 425 |
| HCC direct tipping | 0 | 0 | 0 | 0 | 0 |
| ELWA | 50 | 50 | 50 | 50 | 250 |
| Shanks SRFF | 0 | 0 | 0 | 0 | 0 |
| Other third party (S) | 60 | 60 | 60 | 60 | 300 |
| Countrystyle | 0 | 0 | 0 | 0 | 0 |
| 18-01-04. | 2 | 2 | 2 | 2 | 9 |
| Clinical. | 0 | 0 | 0 | 0 | 0 |
| Others. | 4 | 4 | 4 | 4 | 20 |
| Others. | 0 | 0 | 0 | 0 | 0 |
| Planned Inputs | 2,361 | 2,311 | 2,261 | 2,361 | 11,954 |
| Actual Inputs | 0 | 0 | 0 | 0 | 0 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Stock (daytime peak) | 5,261 | 5,922 | 6,533 | 7,243 |
|
|
|
|
|
|
|
|
| Stock (closing balance) | 3,611 | 4,272 | 4,883 | 5,593 |
|
| Stock (closing balance) | 2,900 | 0 | 0 | 0 | 0 |
| Stock movement +/- | 711 | 661 | 611 | 711 | 404 |
| Stock movement +/- | 0 | 0 | 0 | 0 | 0 |
And I want this to be changed into the below format whenever they make a change in the data i.e., every day
Actual | 02/01/2020 | 1 | 1100 | 0 | 1600 | 1600 | 1800 | 0 | 10000 | 0 | 1681 | 6201 | 460 | 602 | 290 | 833 | 0 | 92 | 0 | 0 | 2 | 1 | 2280 | 0 | 6800 | 599 |
Plan | 03/01/2020 | 1 | 330 | 330 | 330 | 330 | 330 | 42 | 0 | 1650 | 0 | 6822 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 6822 | 5172 | -1650 |
Actual | 03/01/2020 | 1 | 1600 | 700 | 1200 | 1500 | 1700 | 0 | 10000 | 0 | 1707 | 6800 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 5093 | -1707 |
Plan | 04/01/2020 | 1 | 330 | 330 | 330 | 330 | 330 | 42 | 0 | 1650 | 0 | 5172 | 500 | 600 | 300 | 800 | 0 | 80 | 0 | 0 | 1.5 | 4 | 2285.5 | 7457.5 | 5807.5 | 635.5 |
Actual | 04/01/2020 | 1 | 1500 | 700 | 0 | 1400 | 1600 | 0 | 10000 | 0 | 1663 | 5093 | 446 | 465 | 271 | 857 | 0 | 105 | 0 | 0 | 2 | 1 | 2147 | 0 | 5577 | 484 |
Plan | 05/01/2020 | 1 | 330 | 330 | 330 | 330 | 330 | 42 | 0 | 1650 | 0 | 5807.5 | 500 | 600 | 200 | 800 | 0 | 80 | 0 | 0 | 1.5 | 4 | 2185.5 | 7993 | 6343 | 535.5 |
Actual | 05/01/2020 | 1 | 1700 | 1400 | 0 | 1600 | 1000 | 0 | 10000 | 0 | 1632 | 5577 | 916 | 417 | 147 | 869 | 0 | 93 | 0 | 0 | 2 | 1 | 2445 | 0 | 6390 | 813 |
Plan | 06/01/2020 | 2 | 330 | 330 | 330 | 330 | 330 | 42 | 0 | 1650 | 0 | 6343 | 500 | 0 | 0 | 600 | 0 | 80 | 0 | 0 | 2 | 4 | 1186 | 7529 | 5879 | -464 |
Actual | 06/01/2020 | 2 | 1500 | 1600 | 1000 | 1200 | 1200 | 0 | 10000 | 0 | 1654 | 6390 | 515 | 0 | 0 | 750 | 0 | 66 | 0 | 0 | 0 | 0 | 1331 | 0 | 6067 | -323 |
Regards,
Mohammad
Hello Mohammada
I suppose you are reading from a pivottable. This is always tricky and needs some specific techniques to find a result. In my proposed solution i first identify the planned columns and the actual columns. Then the tables are separated and applied some steps liks removing tops rows, promote headers and and unpivot inclusive filter. Then both are again combines and pivoted
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("pVZfb9owEP8qFk+d5lLHiQM8Ulq0SV2HSqc+VH1wE9NEBDtynGp8m32WfbLZToCM2INtknPOkbvf/fHdmefnweJuen9/ezOAg/6azh6/Te+Wxx9e4LPZERkGaIgRRoaJu8yoy4w7zH79izKa7JnWhS+Ca/pYM02fWGres9pQoWhR7Y35xOYyt0jXIi+YBCnbUJ6CCyU4Z9UHZ0qua77WopUSyRpQpV+oVECsQEq34OI9r2pagCRjybqn37EU/PyhfwkxcdLfLAUm8N7TwcLnIuFTSOG5SOEppOj8+KJTWG4MBxLxItly0FwQWy3n1iwrCV47h1z5UTMp6resrBUoRKWAEqDmZUF18ZgSi3CfuFdbzP31hX4f7pxJaEmTXG2Nx8hJDMyiMa+VpCn5AMbEuwWQkL90aJqo2uZHcrDK32rJvNnxRnts6Kbpug1TYCXFxil4QsbALG1HXoiS8Zy/gVeqM5Ew04QYTkzBwDgwvRRBPLLnAcfjcA9xSrsf32euD75yeuv+ZEv6YXY1Xyw0SwhyUgxH7aEcZN0J/iQkr9gWLJVkTA1NSCPkpBiGLWZfx4PNeCq4+R5EyElHqB3nB1E31P3d0xSkuWSJ7pC8LHV6jTo5gMTNu6VhU5t6ufU87s5m4IkqJlNaMHAFDK+HgtI6KyFNM+ImuT2qe6ANowvxh8RosZ5TY9InEe4AnxnGrQ7ZFALqk8ZbU6gZ5esKLB/mcy/OV5XpgaGyXKag1NeTvpiWppRj1Cfh7hxnQidMbiu1LfxNHYwvUXCJIpMgfPRMWpwi53lCC38OrXuVLdijZ/cX4SDhRthNuX2j6QqPg2YPmh3veLvrWTch0aA7w/a6/zm+2smhr3+VbxgoGbU3P2k9IHCCTXZiSEIzbkYQR+E5gIm+U45GkW+EEUgmvVHm0PeNskZjI97ZhnEFPl5dGketqdgG0ZhtfolQ1LVzpOVI5ssv", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t, Column3 = _t, Column4 = _t, Column5 = _t, Column6 = _t, Column7 = _t, Column8 = _t, Column9 = _t, Column10 = _t, Column11 = _t, Column12 = _t, Column13 = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}, {"Column8", type text}, {"Column9", type text}, {"Column10", type text}, {"Column11", type text}, {"Column12", type text}, {"Column13", type text}}), IdentifyColumnWithCellActual = List.Select ( List.Transform ( Table.ColumnNames(#"Changed Type"), (trans)=> if List.Count(List.Select(Table.Column(#"Changed Type", trans),each _ = "ACTUALS" ))>0 then trans else null ), each _ <> null ){0}, IdentifyPlannedColumns = List.FirstN ( Table.ColumnNames(#"Changed Type"), each _ <> IdentifyColumnWithCellActual ), IdentifyActualColumns = List.Difference ( Table.ColumnNames(#"Changed Type"), IdentifyPlannedColumns ), CreatePlannedTable = let SelColumn = Table.SelectColumns(#"Changed Type", IdentifyPlannedColumns), RemoveTop1 = Table.Skip(SelColumn,1), PromoteHeaders = Table.PromoteHeaders(RemoveTop1, [PromoteAllScalars=true]), RemoveTop2 = Table.Skip(PromoteHeaders,1), UnpivotOtherThenFirst = Table.UnpivotOtherColumns(RemoveTop2, {""}, "Attribute", "Value"), TableSelectRows = Table.SelectRows ( UnpivotOtherThenFirst, each Value.Is(try Date.From(_[Attribute]) otherwise "", type date) and _[Value]<>"" ), AddPlan = Table.AddColumn(TableSelectRows, "Type", each "Planned") in AddPlan, CreateActualTable = let SelColumn = Table.SelectColumns(#"Changed Type", IdentifyActualColumns), RemoveTop1 = Table.Skip(SelColumn,1), PromoteHeaders = Table.PromoteHeaders(RemoveTop1, [PromoteAllScalars=true]), RemoveTop2 = Table.Skip(PromoteHeaders,1), UnpivotOtherThenFirst = Table.UnpivotOtherColumns(RemoveTop2, {""}, "Attribute", "Value"), TableSelectRows = Table.SelectRows ( UnpivotOtherThenFirst, each Value.Is(try Date.From(_[Attribute]) otherwise "", type date) and _[Value]<>"" ), AddActual = Table.AddColumn(TableSelectRows, "Type", each "Actual") in AddActual, CombineBoth =Table.Combine({CreatePlannedTable, CreateActualTable}), #"Changed Type1" = Table.TransformColumnTypes(CombineBoth,{{"Value", type number}}), #"Pivoted Column" = Table.Pivot(#"Changed Type1", List.Distinct(#"Changed Type1"[#""]), "", "Value", List.Sum) in #"Pivoted Column"Copy paste this code to the advanced editor in a new blank query to see how the solution works.
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
JimmyHello Mohammada
if the code worked properly I would ask you to mark it as solution. About your request of Mail, you could use Exchange.Contents to read your Mailbox and in case apply filters according to your logic (sender, time, subject) and then read the according appendix
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy
11 Replies
- Jimmy801
Community Champion
Hello Mohammada
I suppose you are reading from a pivottable. This is always tricky and needs some specific techniques to find a result. In my proposed solution i first identify the planned columns and the actual columns. Then the tables are separated and applied some steps liks removing tops rows, promote headers and and unpivot inclusive filter. Then both are again combines and pivoted
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("pVZfb9owEP8qFk+d5lLHiQM8Ulq0SV2HSqc+VH1wE9NEBDtynGp8m32WfbLZToCM2INtknPOkbvf/fHdmefnweJuen9/ezOAg/6azh6/Te+Wxx9e4LPZERkGaIgRRoaJu8yoy4w7zH79izKa7JnWhS+Ca/pYM02fWGres9pQoWhR7Y35xOYyt0jXIi+YBCnbUJ6CCyU4Z9UHZ0qua77WopUSyRpQpV+oVECsQEq34OI9r2pagCRjybqn37EU/PyhfwkxcdLfLAUm8N7TwcLnIuFTSOG5SOEppOj8+KJTWG4MBxLxItly0FwQWy3n1iwrCV47h1z5UTMp6resrBUoRKWAEqDmZUF18ZgSi3CfuFdbzP31hX4f7pxJaEmTXG2Nx8hJDMyiMa+VpCn5AMbEuwWQkL90aJqo2uZHcrDK32rJvNnxRnts6Kbpug1TYCXFxil4QsbALG1HXoiS8Zy/gVeqM5Ew04QYTkzBwDgwvRRBPLLnAcfjcA9xSrsf32euD75yeuv+ZEv6YXY1Xyw0SwhyUgxH7aEcZN0J/iQkr9gWLJVkTA1NSCPkpBiGLWZfx4PNeCq4+R5EyElHqB3nB1E31P3d0xSkuWSJ7pC8LHV6jTo5gMTNu6VhU5t6ufU87s5m4IkqJlNaMHAFDK+HgtI6KyFNM+ImuT2qe6ANowvxh8RosZ5TY9InEe4AnxnGrQ7ZFALqk8ZbU6gZ5esKLB/mcy/OV5XpgaGyXKag1NeTvpiWppRj1Cfh7hxnQidMbiu1LfxNHYwvUXCJIpMgfPRMWpwi53lCC38OrXuVLdijZ/cX4SDhRthNuX2j6QqPg2YPmh3veLvrWTch0aA7w/a6/zm+2smhr3+VbxgoGbU3P2k9IHCCTXZiSEIzbkYQR+E5gIm+U45GkW+EEUgmvVHm0PeNskZjI97ZhnEFPl5dGketqdgG0ZhtfolQ1LVzpOVI5ssv", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t, Column3 = _t, Column4 = _t, Column5 = _t, Column6 = _t, Column7 = _t, Column8 = _t, Column9 = _t, Column10 = _t, Column11 = _t, Column12 = _t, Column13 = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}, {"Column8", type text}, {"Column9", type text}, {"Column10", type text}, {"Column11", type text}, {"Column12", type text}, {"Column13", type text}}), IdentifyColumnWithCellActual = List.Select ( List.Transform ( Table.ColumnNames(#"Changed Type"), (trans)=> if List.Count(List.Select(Table.Column(#"Changed Type", trans),each _ = "ACTUALS" ))>0 then trans else null ), each _ <> null ){0}, IdentifyPlannedColumns = List.FirstN ( Table.ColumnNames(#"Changed Type"), each _ <> IdentifyColumnWithCellActual ), IdentifyActualColumns = List.Difference ( Table.ColumnNames(#"Changed Type"), IdentifyPlannedColumns ), CreatePlannedTable = let SelColumn = Table.SelectColumns(#"Changed Type", IdentifyPlannedColumns), RemoveTop1 = Table.Skip(SelColumn,1), PromoteHeaders = Table.PromoteHeaders(RemoveTop1, [PromoteAllScalars=true]), RemoveTop2 = Table.Skip(PromoteHeaders,1), UnpivotOtherThenFirst = Table.UnpivotOtherColumns(RemoveTop2, {""}, "Attribute", "Value"), TableSelectRows = Table.SelectRows ( UnpivotOtherThenFirst, each Value.Is(try Date.From(_[Attribute]) otherwise "", type date) and _[Value]<>"" ), AddPlan = Table.AddColumn(TableSelectRows, "Type", each "Planned") in AddPlan, CreateActualTable = let SelColumn = Table.SelectColumns(#"Changed Type", IdentifyActualColumns), RemoveTop1 = Table.Skip(SelColumn,1), PromoteHeaders = Table.PromoteHeaders(RemoveTop1, [PromoteAllScalars=true]), RemoveTop2 = Table.Skip(PromoteHeaders,1), UnpivotOtherThenFirst = Table.UnpivotOtherColumns(RemoveTop2, {""}, "Attribute", "Value"), TableSelectRows = Table.SelectRows ( UnpivotOtherThenFirst, each Value.Is(try Date.From(_[Attribute]) otherwise "", type date) and _[Value]<>"" ), AddActual = Table.AddColumn(TableSelectRows, "Type", each "Actual") in AddActual, CombineBoth =Table.Combine({CreatePlannedTable, CreateActualTable}), #"Changed Type1" = Table.TransformColumnTypes(CombineBoth,{{"Value", type number}}), #"Pivoted Column" = Table.Pivot(#"Changed Type1", List.Distinct(#"Changed Type1"[#""]), "", "Value", List.Sum) in #"Pivoted Column"Copy paste this code to the advanced editor in a new blank query to see how the solution works.
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy- MohammadaRegular Visitor
Hi Jimmy,
Thank you so much for your prompt response,much appreciated.
The code worked however I don't have the static data. the data comes in an email every day in the excel format. Is there a way I can automate to look at "X" file and it updates the data in BI.
- Jimmy801
Community Champion
Hello Mohammada
if the code worked properly I would ask you to mark it as solution. About your request of Mail, you could use Exchange.Contents to read your Mailbox and in case apply filters according to your logic (sender, time, subject) and then read the according appendix
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy