Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
MladenJ
Frequent Visitor

Count number of records in a column for all columns

Hi,

 

I'm trying to evaluate all columns in one big table (99 columns, 1.5m rows).

What i need get is the list of all columns and the count of how many records are there in the respective column.

Table has various column types: dates, strings, numbers, boolean values.

 

What is the best approach to do this?

 

What i've tried so far are List functions, but it either seems too long for the refresh (over an hour and a half).

My first step was to get the column names from the fact table and second step is to count the records in selected column as list.

 

let
    Table = data,
    Source = Table.FromList(Table.ColumnNames(Table), Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    RecordCount = Table.AddColumn(Source, "Record count", each try List.NonNullCount( List.ReplaceMatchingItems( Table.ToList(Table.SelectColumns(Table, [Column1])), {{"", null}} )) otherwise 
List.Count( Table.ToList(Table.SelectColumns(Table, [Column1])) ))
in
    RecordCount

 

 

I found that the List.Count or List.NonNullCount don't work as expected, as i have nulls in the list, but it counts them either way.

1 ACCEPTED SOLUTION
wdx223_Daniel
Super User
Super User

NewStep=Table.FromColumns({Table.ColumNames(PreviousStepName),List.Transform(Table.ToColumns(PrevousStepName),each List.Count(List.RemoveItems(_,{"",null})))},{"ColName","NoNullCount"})

View solution in original post

2 REPLIES 2
wdx223_Daniel
Super User
Super User

NewStep=Table.FromColumns({Table.ColumNames(PreviousStepName),List.Transform(Table.ToColumns(PrevousStepName),each List.Count(List.RemoveItems(_,{"",null})))},{"ColName","NoNullCount"})

Thanks, that works, but it still takes really long time to refresh. (hour for now)

 

I'm not too knowledgeable in PQ, but i've read the posts and use cases for Buffer functions. 

Are they any good in this case?

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

February 2024 Update Carousel

Power BI Monthly Update - February 2024

Check out the February 2024 Power BI update to learn about new features.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.

Top Solution Authors
Top Kudoed Authors