Forum Discussion

Gomathivalli's avatar
Gomathivalli
New Member
7 months ago
Solved

Will Unique include the null's count in column profiling in Power Query Editor?

Just have a question on column profiling in Power Query editor Number(Whole num) Text(Text Format) 1 B 3 C 0 B 4 null A 2 C 5 A This is my table and column profiling shows as follows Number...
  • Olufemi7's avatar
    7 months ago

    Hello Gomathivalli,

     

    In Power Query Editor (both in Power BI Desktop and Microsoft Fabric Dataflows), column profiling does include nulls in the Distinct and Unique counts. Nulls are treated as valid values during profiling, even though they are also reported separately under Empty.

    How profiling metrics work:

    • Count → Total rows, including nulls.
    • Distinct → Number of different values, including null.
    • Unique → Number of values that occur only once, including null if it appears once.
    • Empty → Number of nulls.
    • Empty String → Number of “” values (different from null).


    Example:
    For a numeric column with values 1, 3, 0, 4, null, 2, 5 → Distinct = 7, Unique = 7, Empty = 1.
    For a text column with values A, B, C, null, B, C, A → Distinct = 4, Unique = 1, Empty = 1.

    Power BI Desktop: Nulls are included in Distinct and Unique counts. Profiling pane shows Empty separately to distinguish nulls.
    Microsoft Fabric: Uses the same Power Query engine as Power BI. Profiling results are identical: nulls are included in Distinct and Unique counts.

    Microsoft documentation:

    • Data profiling tools in Power Query: `https://learn.microsoft.com/en-us/power-query/data-profiling-tools
    • Table.Profile (Power Query M): `https://learn.microsoft.com/en-us/powerquery-m/table-profile


    Summary: Nulls are always included in Distinct and Unique counts in Power Query Editor for both Power BI Desktop and Microsoft Fabric. If you want profiling without nulls, you must filter them out first (for example, using Remove Empty).

    Power Query M example to exclude nulls before profiling:

    let
    Source = YourTable,
    NoNulls = Table.SelectRows(Source, each [ColumnName] <> null),
    Profile = Table.Profile(NoNulls)
    in
    Profile


    This way, the profiling results will exclude nulls from Distinct and Unique counts in both Power BI and Fabric.