Forum Discussion
Extraction of multiple 7 characters text substrings starting with "O-" from string of text
Hi,
I have a column [Description] where in each row string of text may contain few order numbers (between 0 and 5 order numbers, and they start with “O-“) that I have to extract. I found starting positions of these orders and have them as a list in each row:
=Table.AddColumn(#"Added Custom", "PositionList", each Text.PositionOf([Description],"O-",Occurrence.All))
Order numbers are random alphanumeric combinations and have fixed length of 7. I need help with extraction of order numbers out of text – Text.Middle would work if I can do a loop and pass the list of starting positions as an argument. I am aiming to replace in my list starting positions with order numbers themselves as later I will need to remove duplicates (with List.Distict) from the output list.
I am certain i am making it more complicated than it needs to be and there is more streamlined way of achieving that☹, but i can't figure it out.
I am grateful for any help and advice.
Vlad
Vladisam there you go
let Source = Web.BrowserContents("https://community.powerbi.com/t5/Power-Query/Extraction-of-multiple-7-characters-text-substrings-starting/m-p/2117172#M62409"), #"Extracted Table From Html" = Html.Table(Source, {{"Column1", "DIV[id='bodyDisplay_8'] > DIV.lia-message-body-content > TABLE:nth-child(3) > * > TR > :nth-child(1)"}}, [RowSelector="DIV[id='bodyDisplay_8'] > DIV.lia-message-body-content > TABLE:nth-child(3) > * > TR"]), #"Changed Type" = Table.TransformColumnTypes(#"Extracted Table From Html",{{"Column1", type text}}), fx = let fx =(input)=> Web.Page( "<script> var x='"&input&"'; var b = x.match(/O[-][0-9A-Za-z]{5}/gm); document.write(b); </script>"){0}[Data]{0}[Children]{1}[Children] in fx, #"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each fx([Column1])), #"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"Text"}, {"Text"}), #"Added Custom1" = Table.AddColumn(#"Expanded Custom", "Custom", each List.Distinct(Text.Split([Text],","))), #"Extracted Values" = Table.TransformColumns(#"Added Custom1", {"Custom", each Text.Combine(List.Transform(_, Text.From), ","), type text}), #"Removed Columns" = Table.RemoveColumns(#"Extracted Values",{"Text"}) in #"Removed Columns"- Anonymous4 years ago
Ok check it out:
=Table.AddColumn("Orders", each List.Select(Text.Split([Column1], " "), each Text.StartsWith(_, "OI-")))
-- Nate
31 Replies
- AnonymousNot applicable
Ok check it out:
=Table.AddColumn("Orders", each List.Select(Text.Split([Column1], " "), each Text.StartsWith(_, "OI-")))
-- Nate
- VladisamHelper II
That works - thanks a lot! So I have two valid solutions for my question - Power BI community is awesome!
For those who will be looking at this post later for their purposes - in answer above replace [Column1] with [Description] and "OI-" with "O-".
- ronrsnfldSuper User
How about providing a representative sample of your data and expected output.
- VladisamHelper II
I should've done that (see my reply to Eyelyn9) but it's customer's data - needs too much obfuscation. I made it work with List functions (also in my reply to Eyelyn9). Will try watkinnc' approach shortly just to have alternative.
- ronrsnfldSuper User
OK, I came up with a similar solution as I see in some of the other answers:
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each List.Distinct( List.Select( Text.Split([Column1]," "), each Text.StartsWith(_,"O-") and Text.Length(_)=7)))
- AnonymousNot applicable
I would try a different approach. If the Order Numbers in the Description column are separated by commas, I would first use the split columns GUI function to split by the comma. If you don't have them separated by comma (or something else, I would use the Replace function to replace "OI" in the Description column with ",OI", and THEN split by comma.
Then, whatever the name of your list column is (let's say it's called "Split"), add your custom column:
Table.AddColumn(PriorStepName, "Orders", each List.Select(List.Distinct([Split]), Text.StartsWith(_, "OI)))
--Nate
- VladisamHelper II
Hi Nate,
Order numbers are embedded into string of text (300-400 characters) without commas. Not sure how many of them can be included (data is new to me, I came across 5 orders so far - and these are actually only 2 unique orders twice + error 🙂 ), so it makes it difficult to built robust solution. I managed to make it work with nested List functions (see response to Eyelyn9), thus avoiding to replicate logic on multiple split columns.
List functions rock :).
- mahoneypatMicrosoft Employee
This would be a good application of using RegEx to extract parts of your text string that match the desired pattern. This link shows how to use Web.Page() to do it, but it can also be done with a little bit of R or Python.
RegEx in Power BI and Power Query in Excel with Java Script – The BIccountant
Pat
- VladisamHelper II
I got away with List functions 🙂 - see reply above.
JS is something completely new for me as business user (but a lot of good stuff there, have to givve it a try at some point). Always nice to have alternative paths to solution. Thanks.
- smpa01Community Champion
Vladisam just so you know, for more complex scenarios, regex might be the only choice left and I am so glad that regex can be natively run IN PQ.
Here is how that can be applied in your case.
Let's suppose your dataset is following
| Column1 | |-----------------------------------------------------| | LorempsumdolorsitametO123456eturadipiscingelitseddo | | LorempsumdolorsitametO123456eturadipiscingelitseddo | | LorempsumdolorsitametO12345turadipiscingelitseddo | | LorempsumdolorsitametOabcd12piscingelitseddo | | LorempsumdolorsitametOabcdpiscingelitseddo |You want to extract exact 7 consecutive characters in length dynamically anywhere from the string starting with O
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W8skvSs0tKC7NTcnPyS8qzixJzE0t8Tc0MjYxNUstKS1KTMksyCxOzsxLT83JLClOTUnJV4rVwaIvCQrI0A/XQbx9ChAAdW5iUnKKoRFx7oRrwFQeCwA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}), fx = let fx =(input)=> Web.Page( "<script> var x='"&input&"'; var b = x.match(/O[0-9A-Za-z]{6}/gm); document.write(b); </script>"){0}[Data]{0}[Children]{1}[Children] in fx, #"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each fx([Column1])), #"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"Text"}, {"Text"}) in #"Expanded Custom"- VladisamHelper II
Totally agree - that seems to be very powerful. (BTW can it handle multiple substrings within same string?) I am lucky (for now) that substrings i need have spaces on both sides, but my users are unpredictable, so i am walking on thin ice.
This is waaay above my current level 😞 but definitely need learning.
And I need it in Service and I am not friends with R or Pythoon :(.
- AnonymousNot applicable
Hi Vladisam ,
From this:
between 0 and 5 order numbers, and they start with “O-“
Order numbers are random alphanumeric combinations and have fixed length of 7.I built a simply data sample:
Then split the column by Custom Delimiter like this:
Output:
If it is not your expected, please share more detail information to help us clarify your scenario.
Best Regards,
Eyelyn QinIf this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
- VladisamHelper II
Sorry I think I should've shared the sample. My challenge was to extract only unique values from the list, which is a column in the table (or get only unique values into the list in the first place). I got this to work:
Table.AddColumn(#"Added Custom", "Try1", each List.Distinct(List.FindText(Text.Split([Description]," "),"O-")))Thanks for reply, though.
- AnonymousNot applicable
There's a much simpler way to do this. You can use Replace Values, and first replace "O-" with "~~O-". Next, Replace " " with "--".
Now you can Text.BetweenDelimiters using. "~~" as the delimiter.
--Nate
- VladisamHelper II
Very elegant solution, however picks up only the first order - how do I pick up multiple ones and remove dupes?