Forum Discussion

srinrealyf's avatar
srinrealyf
Regular Visitor
10 months ago
Solved

Issue: Power BI not recognizing null or blank values in few specific columns

I’m facing an issue where the column_a column in my Power BI table doesn’t show any null or blank values in the Data View filter dropdown, even though nulls are clearly visible in visuals.

 

I’ve tried several approaches to identify or count blanks, including:

  • Using DAX measures with ISBLANK() and TRIM()

  • Creating a custom column in Power Query with conditions like
    if [column_a] = null or Text.Trim(Text.From([column_a])) = "" then true else false

  • Also tried findind the length of characters less than 2 to find if its a empty space
  • Checking for nulls directly in the data source (nulls are present in datasource)

Despite these, the null rows don’t appear or behave correctly in Power BI. Could you please help me understand why the column_a and few similar columns behaves differently from other columns where blank detection works as expected, and how to correctly detect or display these null values?

 

Note: The data type is Text and no relationship is used in the model

 

 

  • Hi srinrealyf,

    Did you check for Non-Printable Characters?

    The most likely culprit is that your "null" values contain non-printable characters like spaces, tabs, or other whitespace, you can try this to fix this issue:

    = Table.AddColumn(#"Previous Step", "CheckBlanks", each 
        let
            trimmed = Text.Trim(Text.From([column_a])),
            clean = Text.Clean(trimmed),
            isActuallyBlank = 
                [column_a] = null 
                or trimmed = "" 
                or clean = "" 
                or Text.Length(clean) = 0
        in
            isActuallyBlank
    )

     

    Second Approach: 

    • Create a custom column that checks for all possible blank scenarios:

    = Table.AddColumn(#"Previous Step", "IsActuallyBlank", each 
        let
            textValue = Text.From([column_a]),
            cleanValue = Text.Clean(Text.Trim(textValue))
        in
            if [column_a] = null then "Null"
            else if textValue = "" then "Empty String"
            else if cleanValue = "" then "Whitespace"
            else if Text.Length(cleanValue) = 0 then "Zero Length"
            else "Has Value: " & textValue
    )
    • Or you can try these more comprehensive DAX measures:
    Blank Count Comprehensive =
    VAR CleanValue = TRIM(REPLACE('Table'[column_a], UNICHAR(160), UNICHAR(32)))  // Replace non-breaking spaces
    RETURN
        CALCULATE(
            COUNTROWS('Table'),
            'Table'[column_a] = BLANK() ||
            ISBLANK('Table'[column_a]) ||
            CleanValue = "" ||
            LEN(CleanValue) = 0
        )
    
    // Alternative approach using FILTER
    Blank Detection = 
    COUNTROWS(
        FILTER(
            'Table',
            ISBLANK('Table'[column_a]) ||
            TRIM('Table'[column_a]) = "" ||
            LEN(TRIM('Table'[column_a])) = 0
        )
    )

     

    Third Approach : 

    • Add these transformation steps in Power Query:
    // Step 1: replace various null-like values
    = Table.ReplaceValue(#"Previous Step",each [column_a],each if [column_a] = null then null else Text.Trim(Text.Clean(Text.From([column_a]))),Replacer.ReplaceValue,{"column_a"})
    
    // Step 2: replace empty strings with null
    = Table.ReplaceValue(#"Previous Step","",null,Replacer.ReplaceValue,{"column_a"})
    
    // Step 3: additional cleaning for special whitespace
    = Table.TransformColumns(#"Previous Step", {{"column_a", 
        each if _ = null then null 
        else Text.Trim(Text.Clean(Text.Replace(_, "[^\u0000-\u007F]+", " ")))
    }})

     

    For quick analysis create a calculated column that explicitly marks the problematic values:

    Clean Column = 
    IF(
        ISBLANK('Table'[column_a]) || 
        TRIM('Table'[column_a]) = "" || 
        LEN(TRIM('Table'[column_a])) = 0,
        "BLANK",
        'Table'[column_a]
    )

     

    You can create a calculated table for testing:

    Test Table = 
    SELECTCOLUMNS(
        FILTER(
            'Your Table',
            ISBLANK('Your Table'[column_a]) || 
            TRIM('Your Table'[column_a]) = ""
        ),
        "Blank Values", 'Your Table'[column_a],
        "IsBlank Result", ISBLANK('Your Table'[column_a]),
        "Trim Length", LEN(TRIM('Your Table'[column_a]))
    )

     

    Also don't Forget to check the Data Source:

    • Add a diagnostic column to see what's actually in your data:
    = Table.AddColumn(#"Previous Step", "Debug_column_a", each 
        let
            raw = [column_a],
            asText = Text.From(raw),
            charCodes = Text.ToList(asText),
            codes = List.Transform(charCodes, each Character.ToNumber(_))
        in
            [
                RawValue = raw,
                AsText = asText,
                Length = Text.Length(asText),
                CharacterCodes = codes,
                IsNull = raw = null
            ]
    )

     

    if this post helps, then I would appreciate a thumbs up and mark it as the solution to help the other members find it more quickly.

5 Replies

  • Hi srinrealyf,

    Did you check for Non-Printable Characters?

    The most likely culprit is that your "null" values contain non-printable characters like spaces, tabs, or other whitespace, you can try this to fix this issue:

    = Table.AddColumn(#"Previous Step", "CheckBlanks", each 
        let
            trimmed = Text.Trim(Text.From([column_a])),
            clean = Text.Clean(trimmed),
            isActuallyBlank = 
                [column_a] = null 
                or trimmed = "" 
                or clean = "" 
                or Text.Length(clean) = 0
        in
            isActuallyBlank
    )

     

    Second Approach: 

    • Create a custom column that checks for all possible blank scenarios:

    = Table.AddColumn(#"Previous Step", "IsActuallyBlank", each 
        let
            textValue = Text.From([column_a]),
            cleanValue = Text.Clean(Text.Trim(textValue))
        in
            if [column_a] = null then "Null"
            else if textValue = "" then "Empty String"
            else if cleanValue = "" then "Whitespace"
            else if Text.Length(cleanValue) = 0 then "Zero Length"
            else "Has Value: " & textValue
    )
    • Or you can try these more comprehensive DAX measures:
    Blank Count Comprehensive =
    VAR CleanValue = TRIM(REPLACE('Table'[column_a], UNICHAR(160), UNICHAR(32)))  // Replace non-breaking spaces
    RETURN
        CALCULATE(
            COUNTROWS('Table'),
            'Table'[column_a] = BLANK() ||
            ISBLANK('Table'[column_a]) ||
            CleanValue = "" ||
            LEN(CleanValue) = 0
        )
    
    // Alternative approach using FILTER
    Blank Detection = 
    COUNTROWS(
        FILTER(
            'Table',
            ISBLANK('Table'[column_a]) ||
            TRIM('Table'[column_a]) = "" ||
            LEN(TRIM('Table'[column_a])) = 0
        )
    )

     

    Third Approach : 

    • Add these transformation steps in Power Query:
    // Step 1: replace various null-like values
    = Table.ReplaceValue(#"Previous Step",each [column_a],each if [column_a] = null then null else Text.Trim(Text.Clean(Text.From([column_a]))),Replacer.ReplaceValue,{"column_a"})
    
    // Step 2: replace empty strings with null
    = Table.ReplaceValue(#"Previous Step","",null,Replacer.ReplaceValue,{"column_a"})
    
    // Step 3: additional cleaning for special whitespace
    = Table.TransformColumns(#"Previous Step", {{"column_a", 
        each if _ = null then null 
        else Text.Trim(Text.Clean(Text.Replace(_, "[^\u0000-\u007F]+", " ")))
    }})

     

    For quick analysis create a calculated column that explicitly marks the problematic values:

    Clean Column = 
    IF(
        ISBLANK('Table'[column_a]) || 
        TRIM('Table'[column_a]) = "" || 
        LEN(TRIM('Table'[column_a])) = 0,
        "BLANK",
        'Table'[column_a]
    )

     

    You can create a calculated table for testing:

    Test Table = 
    SELECTCOLUMNS(
        FILTER(
            'Your Table',
            ISBLANK('Your Table'[column_a]) || 
            TRIM('Your Table'[column_a]) = ""
        ),
        "Blank Values", 'Your Table'[column_a],
        "IsBlank Result", ISBLANK('Your Table'[column_a]),
        "Trim Length", LEN(TRIM('Your Table'[column_a]))
    )

     

    Also don't Forget to check the Data Source:

    • Add a diagnostic column to see what's actually in your data:
    = Table.AddColumn(#"Previous Step", "Debug_column_a", each 
        let
            raw = [column_a],
            asText = Text.From(raw),
            charCodes = Text.ToList(asText),
            codes = List.Transform(charCodes, each Character.ToNumber(_))
        in
            [
                RawValue = raw,
                AsText = asText,
                Length = Text.Length(asText),
                CharacterCodes = codes,
                IsNull = raw = null
            ]
    )

     

    if this post helps, then I would appreciate a thumbs up and mark it as the solution to help the other members find it more quickly.
    • srinrealyf's avatar
      srinrealyf
      Regular Visitor

      Hi Ahmed-Elfeel thank you for all the inputs, this helped me to confirm the presence of null values in my data. Unfortunately the whole issue was in the data load process (I didn't remove any nulls, confirmed with transformation history steps). All I had to do was re-load data from databricks and establish a new connection. 

  • Hi srinrealyf,

    Thank you for reaching out to the Microsoft Fabric Community Forum. Also, thanks to Ahmed-Elfeelparry2k, for those inputs on this thread. 

    Has your issue been resolved? If the response provided by the community member Ahmed-Elfeel, parry2k, addressed your query, could you please confirm? It helps us ensure that the solutions provided are effective and beneficial for everyone.

    Hope this helps clarify things and let me know what you find after giving these steps a try happy to help you investigate this further.

    Thank you for using the Microsoft Community Forum.

    • v-kpoloju-msft's avatar
      v-kpoloju-msft
      Icon for Community Support rankCommunity Support

      Hi srinrealyf,

      Just wanted to follow up. If the shared guidance worked for you, that’s wonderful hopefully it also helps others looking for similar answers. If there’s anything else you'd like to explore or clarify, don’t hesitate to reach out.

      Thank you.