Forum Discussion

afaherty's avatar
afaherty
Helper V
4 months ago
Solved

Extracting complicated text from a column

Greetings. I am trying to figure out how to extract text from a column which is proving to be more complicated than I anticipated. Here is some of the sample data from that column. I need to extract InTASC (or INTASC) plus the number/letter after. 

 

Current DataData Needed
[NAEYC 2010 5a, NAEYC 2020 5a, InTASC 4l]InTASC 4l
[NAEYC 2020 6a](blank)
[NAEYC 2020 6e, InTASC 10j]InTASC 10j
[InTASC 10m, NAEYC 2020 3d, ISTE 2.4.d.]InTASC 10m
[ISTE 2.3.a., INTASC 2m]INTASC 2m
[NAEYC 2020 1b, InTASC 1e, ISTE 2.5.a.]InTASC 1e

 

I am open to any method - DAX, M Code, etc. Thank you!

  • Hi afaherty 
    Thank you for reaching out to the Microsoft Fabric community forum.

    You can do this natively in Power Query without Python by splitting the list into tokens, trimming, then filtering only the items that start with INTASC/InTASC (case-insensitive). This also handles rows with multiple INTASC values and returns them joined with |.

    Add a custom column with:
     

    each
        let
            t = [Current Data],
            cleaned0 =
                if t = null or Text.Trim(t) = ""
                then null
                else t,
            cleaned =
                if cleaned0 = null
                then null
                else Text.Replace(
                        Text.Replace(
                            Text.Replace(
                                Text.Replace(cleaned0, "[", ""),
                            "]", ""),
                        "(", ""),
                    ")", ""),
            parts =
                if cleaned = null
                then {}
                else List.Transform(
                        Splitter.SplitTextByAnyDelimiter({",", ";"}, QuoteStyle.Csv)(cleaned),
                        each Text.Trim(_)
                     ),
            intascOnly =
                List.Select(
                    parts,
                    each Text.StartsWith(Text.Trim(_), "INTASC", Comparer.OrdinalIgnoreCase)
                ),
            result =
                if List.Count(intascOnly) = 0
                then null
                else Text.Combine(intascOnly, " | ")
        in
            result

    This will return InTASC 10m | INTASC 61s when multiple values exist, or blank/null if none are present.

    Hope this helps clarify things and let me know what you find after giving these steps a try happy to help you investigate this further.

    Regards,

    Microsoft Fabric Community Support Team



  • Hi afaherty 


    Since the text can contain words before the brackets, the correct approach is to first extract only the content inside [ ... ], then split and filter for InTASC.

     

    Add a custom column with.

    each
        let
            t = [Current Data],
    
            insideBrackets =
                if t = null
                then null
                else try Text.BetweenDelimiters(t, "[", "]") otherwise null,
    
            parts =
                if insideBrackets = null
                then {}
                else List.Transform(
                        Splitter.SplitTextByAnyDelimiter({",", ";"}, QuoteStyle.Csv)(insideBrackets),
                        each Text.Trim(Text.Clean(_))
                     ),
    
            intascOnly =
                List.Select(
                    parts,
                    each Text.StartsWith(_, "INTASC", Comparer.OrdinalIgnoreCase)
                ),
    
            result =
                if List.Count(intascOnly) = 0
                then null
                else Text.Combine(intascOnly, " | ")
    
        in
            result
    


    This will correctly return InTASC 3n | InTASC 8c for multiple matches, InTASC 5b for a single match, or null if no match is found.

    If you have any other questions, feel free to contact us. We're here to help.

     

    Regards,

    Microsoft Fabric Community Support Team.
     

12 Replies

  • AntrikshSharma's avatar
    AntrikshSharma
    Community Champion

    afaherty Try this:

    let
        Source = Excel.CurrentWorkbook(){[ Name = "Table1" ]}[Content],
        Answer = Table.AddColumn (
            Source,
            "Answer",
            ( x ) =>
                [
                    a = x[Current Data],
                    b = Text.Remove ( Text.Remove ( a, "[" ), "]" ),
                    c = Text.Split ( b, ", " ),
                    d = List.Select ( c, ( y ) => Text.StartsWith ( Text.Lower ( y ), "intasc" ) ),
                    e = List.First ( d )
                ][e],
            type nullable text
        )
    in
        Answer

     

  • v-karpurapud's avatar
    v-karpurapud
    Community Support

    Hi afaherty 
    Thank you for reaching out to the Microsoft Fabric community forum.

    You can do this natively in Power Query without Python by splitting the list into tokens, trimming, then filtering only the items that start with INTASC/InTASC (case-insensitive). This also handles rows with multiple INTASC values and returns them joined with |.

    Add a custom column with:
     

    each
        let
            t = [Current Data],
            cleaned0 =
                if t = null or Text.Trim(t) = ""
                then null
                else t,
            cleaned =
                if cleaned0 = null
                then null
                else Text.Replace(
                        Text.Replace(
                            Text.Replace(
                                Text.Replace(cleaned0, "[", ""),
                            "]", ""),
                        "(", ""),
                    ")", ""),
            parts =
                if cleaned = null
                then {}
                else List.Transform(
                        Splitter.SplitTextByAnyDelimiter({",", ";"}, QuoteStyle.Csv)(cleaned),
                        each Text.Trim(_)
                     ),
            intascOnly =
                List.Select(
                    parts,
                    each Text.StartsWith(Text.Trim(_), "INTASC", Comparer.OrdinalIgnoreCase)
                ),
            result =
                if List.Count(intascOnly) = 0
                then null
                else Text.Combine(intascOnly, " | ")
        in
            result

    This will return InTASC 10m | INTASC 61s when multiple values exist, or blank/null if none are present.

    Hope this helps clarify things and let me know what you find after giving these steps a try happy to help you investigate this further.

    Regards,

    Microsoft Fabric Community Support Team



    • afaherty's avatar
      afaherty
      Helper V

      Hi v-karpurapud ,

      I'm afraid I've run into an issue. The formula you so kindly provided is skipping the first InTASC value if there's no comma before it. For example:

       

      Text in columnYour formula is extractingShould extractIssue
      [InTASC 5b, AMLE 3m]nullInTASC 5bIsn't pulling the InTASC because there's no comma before it
      [AMLE 1b, InTASC 2g]InTASC 2gInTASC 2gNo issue, working fine because there is only an InTASC after a comma
      [InTASC 3n, InTASC 8c]InTASC 8cInTASC 3n | InTASC 8cIsn't pulling the first InTASC because there's no comma before it

       

      Any thoughts? Thank you!!

      • v-karpurapud's avatar
        v-karpurapud
        Community Support

        Hi afaherty 

        The issue is caused by hidden or non-printable characters in the first token. When the string starts directly with InTASC, those characters can prevent Text.StartsWith from matching correctly, even after trimming. That’s why the first value is being skipped, while subsequent values (after commas) work fine.

        Applying Text.Clean and normalizing the text before filtering resolves this.

        Add a custom column with:

         

        each
            let
                t = [Current Data],
                cleaned0 =
                    if t = null or Text.Trim(t) = ""
                    then null
                    else t,
                cleaned =
                    if cleaned0 = null
                    then null
                    else Text.Replace(
                            Text.Replace(
                                Text.Replace(
                                    Text.Replace(cleaned0, "[", ""),
                                "]", ""),
                            "(", ""),
                        ")", ""),
                parts =
                    if cleaned = null
                    then {}
                    else List.Transform(
                            Splitter.SplitTextByAnyDelimiter({",", ";"}, QuoteStyle.Csv)(cleaned),
                            each Text.Upper(Text.Trim(Text.Clean(_)))
                         ),
                intascOnly =
                    List.Select(
                        parts,
                        each Text.StartsWith(_, "INTASC")
                    ),
                result =
                    if List.Count(intascOnly) = 0
                    then null
                    else Text.Combine(intascOnly, " | ")
            in
                result


        This ensures the first InTASC value is correctly detected and also continues to handle multiple matches as expected.

        I hope this clarifies things. Let me know what you find after trying these steps. We will assist you further.

        Regards,

        Microsoft Fabric Community Support Team.



  • DAX isn't designed for string manipulation; M doesn't consist of native regex so far. Thus, use embedded python if you're in possession of knowledge of it.

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WivZzdI10VjAyMDRQME3UUYBxjSBcz7wQx2BnBZOcWKVYHSTVQGmzRCxiqXAthgZZUHm4QC6K8cYpQLXBIa4KRnomeil6QI4fWJ2ZYTFMI0TWWC8RIWuUi2mrYRLC1lS4oaZAbUC1sQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Current Data" = _t]),
    
        #"Run Python script" = Python.Execute("df['Extract'] = df['Current Data'].str.findall(r'intasc .+?\b', 34).str.join(' | ')",[df=Source]),
        df = #"Run Python script"{[Name="df"]}[Value]
    in
        df

     

    In fact, Excel formula is already powerful enough to handle it easily.

    • lbendlin's avatar
      lbendlin
      Super User

      Here's a more pedestrian version with "native"  Power Query

       

      let
          Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WivZzdI10VjAyMDRQME3UUYBxjSBcz7wQx2BnBZOcWKVYHSTVQGmzRCxiqXAthgZZUHm4QC6K8cYpQLXBIa4KRnomeil6QI4fWJ2ZYTFMI0TWWC8RIWuUi2mrYRLC1lS4oaZAbUC1sQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Current Data" = _t]),
          #"Added Custom" = Table.AddColumn(
          Source, 
          "Result", 
          each 
            let
              Start = Text.PositionOf(Text.Upper([Current Data]), "INTASC"), 
              Count = Text.PositionOf(Text.Replace(Text.Range([Current Data], Start), "]", ","), ",")
            in
              if Start = - 1 then null else Text.Range([Current Data], Start, Count)
        )
      in
        #"Added Custom"
      • afaherty's avatar
        afaherty
        Helper V

        lbendlin Thank you. I think something is going awry however. It is pulling in incorrect values into the new custom column (example below). Also, I apologize but I neglected to mention that some rows have more than 1 "InTASC" within them. For example: [InTASC 10m, NAEYC 2020 3d, ISTC 2.4.d., INTASC 61s]

         

        Here is an example of the problem that is now happening which can be seen in the attached file. 

         

        Current DataCustom.Current DataCustom.Result
        [CEC 2020 2.1][NAEYC 2010 5a, NAEYC 2020 5a, InTASC 4l]InTASC 4l
        [CEC 2020 2.1][NAEYC 2020 6a] 
        [CEC 2020 2.1][NAEYC 2020 6e, InTASC 10j]InTASC 10j
        [CEC 2020 2.1][InTASC 10m, NAEYC 2020 3d, ISTE 2.4.d., INTASC 61s]InTASC 10m
        [CEC 2020 2.1][ISTE 2.3.a., INTASC 2m]INTASC 2m
        [CEC 2020 2.1][NAEYC 2020 1b, InTASC 1e, ISTE 2.5.a.]InTASC 1e

         

        Thanks again!