Forum Discussion
need help - Power query or M code solution needed
- 1 year ago
Hi ashwinkolte,
I hope this information is helpful. Please let me know if you have any further questions or if you'd like to discuss this further. If this answers your question, please Accept it as a solution and give it a 'Kudos' so others can find it easily.
Thank you.
GroupKind.Local should be able to help you here.
Create a placeholder column that indicates the blank rows as null else a common value.
From there you can group on the placeholder column, choose no aggregation. You will need to add GroupKind.Local as the fifth variable in the Table.Group() function.
Filter out the null rows.
Promote the headers in the nested tables.
Remove the outer placeholder column.
Expand the desired columns.
let
Source =
#table(
{"Column1", "Column2", "Column3", "Column4", "Column5", "Column6"},
{
{"Monitor", "Usergroup", "Monitor_group", "Contact", "", ""},
{"Monitor1", "Usergroup1", "Monitor_group1", "Contact1", "", ""},
{"Monitor2", "Usergroup2", "Monitor_group2", "Contact2", "", ""},
{"","","","","",""},
{"Monitor", "Poller", "Contact", "Monitor_group", "Usergroup", ""},
{"Monitor3", "Poller3", "Contact3", "Monitor_group3", "Usergroup3", ""},
{"Monitor4", "Poller4", "Contact4", "Monitor_group4", "Usergroup4", ""},
{"","","","","",""},
{"Monitor", "Contact", "Usergroup", "Poller", "Monitor_group", "Portnumber"},
{"Monitor5", "Contact5", "Usergroup5", "Poller4", "Monitor_group5", "Port5"},
{"Monitor6", "Contact6", "Usergroup6", "Poller5", "Monitor_group6", "Port6"}
}
),
add_local_group_column =
Table.AddColumn(
Source,
"placeholder",
each if [Column1] = "" then null else "G",
type text
),
local_group =
Table.Group(
add_local_group_column,
{"placeholder"},
{
{"AllRows", each Table.RemoveColumns(_, {"placeholder"}), type table}
},
GroupKind.Local
),
remove_null_rows =
Table.SelectRows(
local_group,
each ([placeholder] = "G")
),
promote_nested_headers =
Table.TransformColumns(
remove_null_rows,
{
{"AllRows", each Table.PromoteHeaders(_, [PromoteAllScalars = true])}
}
),
remove_placeholder =
Table.RemoveColumns(
promote_nested_headers,
{"placeholder"}
),
expand_desired =
Table.ExpandTableColumn(
remove_placeholder,
"AllRows",
{"Monitor", "Usergroup", "Monitor_group", "Contact", "Poller", "Portnumber"}
)
in
expand_desired
Hope this helps.
- ashwinkolte1 year ago
Helper III
HI jgeddes
First of all thank you so much for super fast response. Really appreciate that . I saw the PBIX file and the code . However I see that Column names are hardcoded . As I mentioned earlier the future blocks can have more columns (upto 25) and hence more common columns . Can the code be changed to perform all the column operations dynamically
- lbendlin1 year ago
Super User
Please provide sample data that covers your issue or question completely, in a usable format (not as a screenshot).
Do not include sensitive information. Do not include anything that is unrelated to the issue or question.
Please show the expected outcome based on the sample data you provided.
Need help uploading data? https://community.fabric.microsoft.com/t5/Community-Blog/How-to-provide-sample-data-in-the-Power-BI-Forum/ba-p/963216
Want faster answers? https://community.fabric.microsoft.com/t5/Desktop/How-to-Get-Your-Question-Answered-Quickly/m-p/1447523- ashwinkolte1 year ago
Helper III
Hi lbendlin
Thankyou so much for responding . Really appresiate that
The sample data and expected output is provided below . I dont see an option to attach files hence pasting the inpout file data. Hopefully you should be able to use this by just copying to an excel file . Have provided a image of expected output since I guess it is only for reference. Please note this is only sample data . In the real file there are more than 100000 rows and over 25 columns in total , spread across blocks of data with different number and sequence of columns as depicted in sample. Hence we cannot use hardcoded column names in the solution
Sample data
Monitor Usergroup Monitor_ group Contact Monitor1 Usergroup1 Monitor_group1 contact1 Monitor2 Usergroup2 Monitor_group2 contact2 Monitor Poller Contact Monitor_ group Usergroup Monitor3 Poller3 Contact3 Monitor_group3 Usergroup3 Monitor4 Poller4 Contact4 Monitor_group4 Usergroup4 Monitor Contact Usergroup Poller Monitor_ group Portnumber Monitor5 Contact5 Usergroup5 Poller4 Monitor_group5 Port5 Monitor6 Contact6 Usergroup6 Poller5 Monitor_group6 Port6
Expected output
- jgeddes1 year ago
Super User
Yep.
let Source = #table( {"Column1", "Column2", "Column3", "Column4", "Column5", "Column6"}, { {"Monitor", "Usergroup", "Monitor_group", "Contact", "", ""}, {"Monitor1", "Usergroup1", "Monitor_group1", "Contact1", "", ""}, {"Monitor2", "Usergroup2", "Monitor_group2", "Contact2", "", ""}, {"","","","","",""}, {"Monitor", "Poller", "Contact", "Monitor_group", "Usergroup", ""}, {"Monitor3", "Poller3", "Contact3", "Monitor_group3", "Usergroup3", ""}, {"Monitor4", "Poller4", "Contact4", "Monitor_group4", "Usergroup4", ""}, {"","","","","",""}, {"Monitor", "Contact", "Usergroup", "Poller", "Monitor_group", "Portnumber"}, {"Monitor5", "Contact5", "Usergroup5", "Poller4", "Monitor_group5", "Port5"}, {"Monitor6", "Contact6", "Usergroup6", "Poller5", "Monitor_group6", "Port6"} } ), add_local_group_column = Table.AddColumn( Source, "placeholder", each if [Column1] = "" then null else "G", type text ), local_group = Table.Group( add_local_group_column, {"placeholder"}, { {"AllRows", each Table.RemoveColumns(_, {"placeholder"}), type table} }, GroupKind.Local ), remove_null_rows = Table.SelectRows( local_group, each ([placeholder] = "G") ), remove_nested_blanks = Table.TransformColumns( remove_null_rows, { {"AllRows", each Table.Transpose(Table.SelectRows(Table.Transpose(_), each ([Column1] <> "")))} } ), promote_nested_headers = Table.TransformColumns( remove_nested_blanks, { {"AllRows", each Table.PromoteHeaders(_, [PromoteAllScalars = true])} } ), nested_column_names = Table.AddColumn( promote_nested_headers, "columnNames", each Table.ColumnNames([AllRows]), type list ), distinct_column_names = List.Distinct( List.Combine(nested_column_names[columnNames]) ), remove_columns = Table.RemoveColumns( nested_column_names, {"placeholder", "columnNames"} ), expand_desired = Table.ExpandTableColumn( remove_columns, "AllRows", distinct_column_names ) in expand_desired- ashwinkolte1 year ago
Helper III
Hi jgeddes
Again thanks for superfast response. But I still have the same question . The input data I shared was only sample . The real life file has more than 100000 rows, 25+ columns in total and throusands of data blocks
I see your solution still considers only 6 columns . Maybe I am missing something . Will the above solution work on the real life file ? We cannot code the below portion manually becuase the the number of columns may increase in the future ? Sorry to bug you again
#table( {"Column1", "Column2", "Column3", "Column4", "Column5", "Column6"}, { {"Monitor", "Usergroup", "Monitor_group", "Contact", "", ""}, {"Monitor1", "Usergroup1", "Monitor_group1", "Contact1", "", ""}, {"Monitor2", "Usergroup2", "Monitor_group2", "Contact2", "", ""}, {"","","","","",""}, {"Monitor", "Poller", "Contact", "Monitor_group", "Usergroup", ""}, {"Monitor3", "Poller3", "Contact3", "Monitor_group3", "Usergroup3", ""}, {"Monitor4", "Poller4", "Contact4", "Monitor_group4", "Usergroup4", ""}, {"","","","","",""}, {"Monitor", "Contact", "Usergroup", "Poller", "Monitor_group", "Portnumber"}, {"Monitor5", "Contact5", "Usergroup5", "Poller4", "Monitor_group5", "Port5"}, {"Monitor6", "Contact6", "Usergroup6", "Poller5", "Monitor_group6", "Port6"} } ),