Forum Discussion

Vladisam's avatar
Vladisam
Helper II
4 years ago
Solved

Extraction of multiple 7 characters text substrings starting with "O-" from string of text

Hi,

 

I have a column [Description] where in each row string of text may contain few order numbers (between 0 and 5 order numbers, and they start with “O-“) that I have to extract. I found starting positions of these orders and have them as a list in each row:

 

=Table.AddColumn(#"Added Custom", "PositionList", each Text.PositionOf([Description],"O-",Occurrence.All))

 

Order numbers are random alphanumeric combinations and have fixed length of 7. I need help with extraction of order numbers out of text – Text.Middle would work if I can do a loop and pass the list of starting positions as an argument. I am aiming to replace in my list starting positions with order numbers themselves as later I will need to remove duplicates (with List.Distict) from the output list.

I am certain i am making it more complicated than it needs to be and there is more streamlined way of achieving that☹, but i can't figure it out.
I am grateful for any help and advice.

Vlad

  • smpa01's avatar
    smpa01
    4 years ago

    Vladisam  there you go

     

     

    let
        Source = Web.BrowserContents("https://community.powerbi.com/t5/Power-Query/Extraction-of-multiple-7-characters-text-substrings-starting/m-p/2117172#M62409"),
        #"Extracted Table From Html" = Html.Table(Source, {{"Column1", "DIV[id='bodyDisplay_8'] > DIV.lia-message-body-content > TABLE:nth-child(3) > * > TR > :nth-child(1)"}}, [RowSelector="DIV[id='bodyDisplay_8'] > DIV.lia-message-body-content > TABLE:nth-child(3) > * > TR"]),
        #"Changed Type" = Table.TransformColumnTypes(#"Extracted Table From Html",{{"Column1", type text}}),
        fx = let   fx =(input)=>
        Web.Page(
            "<script>
                var x='"&input&"';
                var b = x.match(/O[-][0-9A-Za-z]{5}/gm);
                document.write(b);
            </script>"){0}[Data]{0}[Children]{1}[Children]    
    in
        fx,
    #"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each fx([Column1])),
        #"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"Text"}, {"Text"}),
        #"Added Custom1" = Table.AddColumn(#"Expanded Custom", "Custom", each List.Distinct(Text.Split([Text],","))),
        #"Extracted Values" = Table.TransformColumns(#"Added Custom1", {"Custom", each Text.Combine(List.Transform(_, Text.From), ","), type text}),
        #"Removed Columns" = Table.RemoveColumns(#"Extracted Values",{"Text"})
        
    in
        #"Removed Columns"

     

     

     

     

  • Anonymous's avatar
    Anonymous
    4 years ago

    Ok check it out:

     

    =Table.AddColumn("Orders", each List.Select(Text.Split([Column1], " "), each Text.StartsWith(_, "OI-")))

     

    -- Nate

31 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Ok check it out:

     

    =Table.AddColumn("Orders", each List.Select(Text.Split([Column1], " "), each Text.StartsWith(_, "OI-")))

     

    -- Nate

    • Vladisam's avatar
      Vladisam
      Helper II

      That works - thanks a lot! So I have two valid solutions for my question - Power BI community is awesome!

      For those who will be looking at this post later for their purposes - in answer above replace [Column1] with [Description] and "OI-" with "O-".

  • How about providing a representative sample of your data and expected output.

    • Vladisam's avatar
      Vladisam
      Helper II

      I should've done that (see my reply to Eyelyn9) but it's customer's data - needs too much obfuscation. I made it work with List functions (also in my reply to Eyelyn9). Will try watkinnc' approach shortly just to have alternative.

      • ronrsnfld's avatar
        ronrsnfld
        Super User

        OK, I came up with a similar solution as I see in some of the other answers:

         

         

            #"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", 
                each List.Distinct(
                        List.Select(
                            Text.Split([Column1]," "), 
                                each Text.StartsWith(_,"O-") and Text.Length(_)=7)))

         

  • Anonymous's avatar
    Anonymous
    Not applicable

    I would try a different approach. If the Order Numbers in the Description column are separated by commas, I would first use the split columns GUI function to split by the comma. If you don't have them separated by comma (or something else, I would use the Replace function to replace "OI" in the Description column with ",OI", and THEN split by comma.

     

    Then, whatever the name of your list column is (let's say it's called "Split"), add your custom column:

     

    Table.AddColumn(PriorStepName, "Orders", each List.Select(List.Distinct([Split]),  Text.StartsWith(_, "OI)))

     

    --Nate 

    • Vladisam's avatar
      Vladisam
      Helper II

      Hi Nate,

      Order numbers are embedded into string of text (300-400 characters) without commas. Not sure how many of them can be included (data is new to me, I came across 5 orders so far - and these are actually only 2 unique orders twice + error 🙂 ), so it makes it difficult to built robust solution. I managed to make it work with nested List functions (see response to Eyelyn9), thus avoiding to replicate logic on multiple split columns.
      List functions rock :). 

    • Vladisam's avatar
      Vladisam
      Helper II

      I got away with List functions 🙂 - see reply above.

      JS is something completely new for me as business user (but a lot of good stuff there, have to givve it a try at some point). Always nice to have alternative paths to solution. Thanks.

    • smpa01's avatar
      smpa01
      Community Champion

      Vladisam  just so you know, for more complex scenarios, regex might be the only choice left and I am so glad that regex can be natively run IN PQ. 

       

      Here is how that can be applied in your case.

      Let's suppose your dataset is following

       

       

      | Column1                                             |
      |-----------------------------------------------------|
      | LorempsumdolorsitametO123456eturadipiscingelitseddo |
      | LorempsumdolorsitametO123456eturadipiscingelitseddo |
      | LorempsumdolorsitametO12345turadipiscingelitseddo   |
      | LorempsumdolorsitametOabcd12piscingelitseddo        |
      | LorempsumdolorsitametOabcdpiscingelitseddo          |

       

       

      You want to extract exact 7 consecutive characters in length dynamically anywhere from the string starting with O

       

       

      let
          Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W8skvSs0tKC7NTcnPyS8qzixJzE0t8Tc0MjYxNUstKS1KTMksyCxOzsxLT83JLClOTUnJV4rVwaIvCQrI0A/XQbx9ChAAdW5iUnKKoRFx7oRrwFQeCwA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
          #"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
          fx = let   fx =(input)=>
          Web.Page(
              "<script>
                  var x='"&input&"';
                  var b = x.match(/O[0-9A-Za-z]{6}/gm);
                  document.write(b);
              </script>"){0}[Data]{0}[Children]{1}[Children]    
      in
          fx,
      #"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each fx([Column1])),
          #"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"Text"}, {"Text"})
          
      in
          #"Expanded Custom"

       

       

       

       

       

       

       

       

       

      • Vladisam's avatar
        Vladisam
        Helper II

        Totally agree - that seems to be very powerful. (BTW can it handle multiple substrings within same string?) I am lucky (for now) that substrings i need have spaces on both sides, but my users are unpredictable, so i am walking on thin ice.

        This is waaay above my current level 😞 but definitely need learning.

        And I need it in Service and I am not friends with R or Pythoon :(.

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi Vladisam ,

     

    From this:

    between 0 and 5 order numbers, and they start with “O-“
    Order numbers are random alphanumeric combinations and have fixed length of 7.

     

    I built a simply data sample:

    Then split the column by Custom Delimiter like this:

    Output:

     

    If it is not your expected, please share more detail information to help us clarify your scenario.

     

    Best Regards,
    Eyelyn Qin

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

     

    • Vladisam's avatar
      Vladisam
      Helper II

      Sorry I think I should've shared the sample. My challenge was to extract only unique values from the list, which is a column in the table (or get only unique values into the list in the first place). I got this to work:
      Table.AddColumn(#"Added Custom", "Try1", each List.Distinct(List.FindText(Text.Split([Description]," "),"O-")))

      Thanks for reply, though.

  • Anonymous's avatar
    Anonymous
    Not applicable

    There's a much simpler way to do this. You can use Replace Values, and first replace "O-" with "~~O-". Next, Replace " " with "--". 

    Now you can Text.BetweenDelimiters using. "~~" as the delimiter. 

    --Nate

    • Vladisam's avatar
      Vladisam
      Helper II

      Very elegant solution, however picks up only the first order - how do I pick up multiple ones and remove dupes?