Forum Discussion
Finding columns with only null values
Hi
I have a lot of columns (customised fields) from O-data in Project Online with only null values.
Is there a way to show these columns - as I need to delete the customised fields in Project Online that we do not use.
Best regards,
Hasse
Hi Anonymous
If you only want to know which columns are null columns, here is another method which doesn't require much M code knowledge.
In below table, Column C and D only contain null values. The table has 3 rows in total and its name is "Query1".
Then in the left-side Queries pane, right click your mouse and select New Query > Blank Query.
You will see a new query appear. In below image, Query2 is the new query. Input the following code in its formula bar. Replace yourTableName with the original table name.
= Table.Profile(yourTableName)You will get some summary statistics of each column in the new table. There is a "Count" column and "NullCount" column. "Count" column means the total rows in this column. "NullCount" means the number of null values in this column. If these two values are the same, it means this column only has null values.
Based on that, you can add a conditional column to compare two values. If they are equal, return "Yes". If not, return "No".
Then you can filter the new column to only have rows that value is "Yes". The most left column will tell you which columns only have null values.
I get this idea from the following blog. If you want to remove null columns in Power Query Editor, you can follow the detailed steps in this blog.
Dynamically Remove Null Columns in Power Query with M code (antmanbi.com)
Best Regards,
Community Support Team _ Jing
If this post helps, please Accept it as Solution to help other members find it.
8 Replies
Hi Anonymous
If you only want to know which columns are null columns, here is another method which doesn't require much M code knowledge.
In below table, Column C and D only contain null values. The table has 3 rows in total and its name is "Query1".
Then in the left-side Queries pane, right click your mouse and select New Query > Blank Query.
You will see a new query appear. In below image, Query2 is the new query. Input the following code in its formula bar. Replace yourTableName with the original table name.
= Table.Profile(yourTableName)You will get some summary statistics of each column in the new table. There is a "Count" column and "NullCount" column. "Count" column means the total rows in this column. "NullCount" means the number of null values in this column. If these two values are the same, it means this column only has null values.
Based on that, you can add a conditional column to compare two values. If they are equal, return "Yes". If not, return "No".
Then you can filter the new column to only have rows that value is "Yes". The most left column will tell you which columns only have null values.
I get this idea from the following blog. If you want to remove null columns in Power Query Editor, you can follow the detailed steps in this blog.
Dynamically Remove Null Columns in Power Query with M code (antmanbi.com)
Best Regards,
Community Support Team _ Jing
If this post helps, please Accept it as Solution to help other members find it.Here's one way:
#"Select Null Columns" = Table.SelectColumns(#"Previous Step", List.RemoveNulls( List.Accumulate( Table.ColumnNames(#"Previous Step"), {}, (state, current)=> state & { if List.NonNullCount(Table.Column(#"Previous Step",current)) = 0 then current else null } ) ) ) in #"Select Null Columns"You would insert it at the point in your existing code where you want to have the "null columns" selected. Change #"Previous Step" to whatever the name of the previous step is in your existing code; and change your next line of code to refer to #"Select Null Columns" instead of to whatever it presently refers to.
Here's another way to do it in the query editor. To see how it works, just create a blank query, open the Advanced Editor and replace the text there with the M code below. The first 3 steps through #"Replaced Value" are just setting up the demo data. You would do the last two steps on your query.
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUTICYggyVoqNBQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [A = _t, B = _t, C = _t, D = _t, E = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"A", Int64.Type}, {"B", Int64.Type}, {"C", type text}, {"D", type text}, {"E", Int64.Type}}), #"Replaced Value" = Table.ReplaceValue(#"Changed Type","",null,Replacer.ReplaceValue,{"C", "D"}), NotNullColumns = List.Select(Table.ColumnNames(#"Replaced Value"), each Record.Field(#"Replaced Value"{0}, _) <> null), KeepNotNullColumns = Table.SelectColumns(#"Replaced Value", NotNullColumns) in KeepNotNullColumnsPat
