Forum Discussion
How to extract data in between specifics simbols.
- 4 years ago
Hi Anonymous ,
Please trake a look at the following. I have explained the steps briefly and added the sample code at the end as well.Assuming this is your first table named "Waypoints"
and your second table named "Station"
1) On table "Waypoints", Extract text after the first delimiter
This will give you your string after the first "-"
2) Extract text before the last delimiter
This will give you your string before the last"-"
Your data is now in the required format for applying further transformations.
3) Convert the "Text After Delimiter" column into a comma separated list.4) Now, on table "Station", perform a group by on column EU and create a list of stations for each value Y or N as shown below
5) Apply filter EU = "Y"
6) On table "Waypoints", perform a fuzzy merge with table "Station" on column "Custom". Set similarity threshold = 0.1 (Very important!)
You will now see that any value in your list of waypoints that matches your list of EU stations will return a match. Everything else will be returned as null7) Replace null with "N" and rename columns. You will then get your final output
Table : Waypointslet Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("LYo7DoAgFATvQu229iD4QPlERAMS7n8NMVjsbCaZWlm5LEdxOaafxksNEc4IEpGzNlXmKXDsJseBRVLvLJGGVvZb163cI85m53BmhlzTeE8QzwHyqhftBQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Waypoints = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Waypoints", type text}}), #"Inserted Text After Delimiter" = Table.AddColumn(#"Changed Type", "Text After Delimiter", each Text.AfterDelimiter([Waypoints], "-"), type text), #"Extracted Text Before Delimiter" = Table.TransformColumns(#"Inserted Text After Delimiter", {{"Text After Delimiter", each Text.BeforeDelimiter(_, "-", {0, RelativePosition.FromEnd}), type text}}), #"Added Custom" = Table.AddColumn(#"Extracted Text Before Delimiter", "Custom", each {Text.Replace([Text After Delimiter], "-" , " , ")}), #"Extracted Values" = Table.TransformColumns(#"Added Custom", {"Custom", each Text.Combine(List.Transform(_, Text.From)), type text}), #"Merged Queries" = Table.FuzzyNestedJoin(#"Extracted Values", {"Custom"}, Station, {"Custom"}, "Station", JoinKind.LeftOuter, [IgnoreCase=true, IgnoreSpace=true, Threshold=0.1]), #"Expanded Station" = Table.ExpandTableColumn(#"Merged Queries", "Station", {"EU"}, {"EU"}), #"Replaced Value" = Table.ReplaceValue(#"Expanded Station",null,"N",Replacer.ReplaceValue,{"EU"}), #"Removed Columns" = Table.RemoveColumns(#"Replaced Value",{"Custom"}), #"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Text After Delimiter", "Transit Waypoints"}, {"EU", "EU Transit"}}) in #"Renamed Columns"Table : Station
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WivQNClFQ0lHyU4rViVby9HPxgHOc/IOD4BxvzwgEx9nFPSgEyIsE83zc3RGaPFx9kNlgZRCer6cZnO3i546wJioQwo4FAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Station Name" = _t, EU = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Station Name", type text}, {"EU", type text}}), #"Grouped Rows" = Table.Group(#"Changed Type", {"EU"}, {{"Count", each _, type table [Station Name=nullable text, EU=nullable text]}}), #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each [Count][Station Name]), #"Extracted Values" = Table.TransformColumns(#"Added Custom", {"Custom", each Text.Combine(List.Transform(_, Text.From), " , "), type text}), #"Removed Columns" = Table.RemoveColumns(#"Extracted Values",{"Count"}), #"Filtered Rows" = Table.SelectRows(#"Removed Columns", each ([EU] = "Y")) in #"Filtered Rows"Kind regards,
Rohit
Please mark this answer as the solution if it resolves your issue.
Appreciate your kudos! 🙂 - 4 years ago
You have to add as a new query first, and then add {0} to the end of the new query, not the original query named "Station"
This is how your new query should look like.let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WivQNClFQ0lHyU4rViVby9HPxgHOc/IOD4BxvzwgEx9nFPSgEyIsE83zc3RGaPFx9kNlgZRCer6cZnO3i546wJioQznb0CXOEmBsLAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Station Name" = _t, EU = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Station Name", type text}, {"EU", type text}}), #"Grouped Rows" = Table.Group(#"Changed Type", {"EU"}, {{"Count", each _, type table [Station Name=nullable text, EU=nullable text]}}), #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each [Count][Station Name]), #"Extracted Values" = Table.TransformColumns(#"Added Custom", {"Custom", each Text.Combine(List.Transform(_, Text.From), " , "), type text}), #"Removed Columns" = Table.RemoveColumns(#"Extracted Values",{"Count"}), #"Filtered Rows" = Table.SelectRows(#"Removed Columns", each ([EU] = "Y")), Custom1 = #"Filtered Rows"[Custom]{0} in Custom1
You can split by delimiter in Power Query to get the first part of your question.
Once by "-" as far left as possible.
Then Split the resulting column again, by "-" as far right as possible.
--
I don't understand what you are asking in the 2nd part of the question. What are you trying to match up with? Maybe you can provide a data sample?
hmm, i will need to look for " delimiter" then, i dont know that function. Maybe you can explain me alittle bit more about this.
Here you have an axemple.
There you have 3 lines of "waypoits" , then i need to take out the first one and the last one.
Then if you see in detal , CDGRT is the only station in EU, so what I need is to : if there is any waypoint that belong to EU, should be tag as Y in column " EU trasnit"
Hope I was clear enough.
Thansk!!
- rohit_singh4 years ago
Solution Sage
Hi Anonymous ,
Please trake a look at the following. I have explained the steps briefly and added the sample code at the end as well.Assuming this is your first table named "Waypoints"
and your second table named "Station"
1) On table "Waypoints", Extract text after the first delimiter
This will give you your string after the first "-"
2) Extract text before the last delimiter
This will give you your string before the last"-"
Your data is now in the required format for applying further transformations.
3) Convert the "Text After Delimiter" column into a comma separated list.4) Now, on table "Station", perform a group by on column EU and create a list of stations for each value Y or N as shown below
5) Apply filter EU = "Y"
6) On table "Waypoints", perform a fuzzy merge with table "Station" on column "Custom". Set similarity threshold = 0.1 (Very important!)
You will now see that any value in your list of waypoints that matches your list of EU stations will return a match. Everything else will be returned as null7) Replace null with "N" and rename columns. You will then get your final output
Table : Waypointslet Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("LYo7DoAgFATvQu229iD4QPlERAMS7n8NMVjsbCaZWlm5LEdxOaafxksNEc4IEpGzNlXmKXDsJseBRVLvLJGGVvZb163cI85m53BmhlzTeE8QzwHyqhftBQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Waypoints = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Waypoints", type text}}), #"Inserted Text After Delimiter" = Table.AddColumn(#"Changed Type", "Text After Delimiter", each Text.AfterDelimiter([Waypoints], "-"), type text), #"Extracted Text Before Delimiter" = Table.TransformColumns(#"Inserted Text After Delimiter", {{"Text After Delimiter", each Text.BeforeDelimiter(_, "-", {0, RelativePosition.FromEnd}), type text}}), #"Added Custom" = Table.AddColumn(#"Extracted Text Before Delimiter", "Custom", each {Text.Replace([Text After Delimiter], "-" , " , ")}), #"Extracted Values" = Table.TransformColumns(#"Added Custom", {"Custom", each Text.Combine(List.Transform(_, Text.From)), type text}), #"Merged Queries" = Table.FuzzyNestedJoin(#"Extracted Values", {"Custom"}, Station, {"Custom"}, "Station", JoinKind.LeftOuter, [IgnoreCase=true, IgnoreSpace=true, Threshold=0.1]), #"Expanded Station" = Table.ExpandTableColumn(#"Merged Queries", "Station", {"EU"}, {"EU"}), #"Replaced Value" = Table.ReplaceValue(#"Expanded Station",null,"N",Replacer.ReplaceValue,{"EU"}), #"Removed Columns" = Table.RemoveColumns(#"Replaced Value",{"Custom"}), #"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Text After Delimiter", "Transit Waypoints"}, {"EU", "EU Transit"}}) in #"Renamed Columns"Table : Station
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WivQNClFQ0lHyU4rViVby9HPxgHOc/IOD4BxvzwgEx9nFPSgEyIsE83zc3RGaPFx9kNlgZRCer6cZnO3i546wJioQwo4FAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Station Name" = _t, EU = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Station Name", type text}, {"EU", type text}}), #"Grouped Rows" = Table.Group(#"Changed Type", {"EU"}, {{"Count", each _, type table [Station Name=nullable text, EU=nullable text]}}), #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each [Count][Station Name]), #"Extracted Values" = Table.TransformColumns(#"Added Custom", {"Custom", each Text.Combine(List.Transform(_, Text.From), " , "), type text}), #"Removed Columns" = Table.RemoveColumns(#"Extracted Values",{"Count"}), #"Filtered Rows" = Table.SelectRows(#"Removed Columns", each ([EU] = "Y")) in #"Filtered Rows"Kind regards,
Rohit
Please mark this answer as the solution if it resolves your issue.
Appreciate your kudos! 🙂- Anonymous4 years agoNot applicable
Thank you very much!! this is amazing....
Let me try all that you did, and will go back to you.
I will still need to lear how to :
-Step 3 how to Convert into a comma separated list.-Step 4 Group by
-Step 6 fuzzy merge.Let me try everything and i will let you know how it went.
Thanks!
- Anonymous4 years agoNot applicable
Sorry would you mind please help me on the following steps ?
-Step 3 how to Convert into a comma separated list. I ve been looking how to doit but there is not a clear way online.
-Step 4 Group by
-Step 6 fuzzy merge.- rohit_singh4 years ago
Solution Sage
Hi Anonymous ,
I already sent you the sample code in my answer yesterday. You just need to copy and paste the M-code for table "Waypoints" into a new blank query and you will be able to see the steps.Kind regards,
Rohit
- Anonymous4 years agoNot applicable
Hi again...
On the step 7 I am reciving these :
I see the result from the merge as a table column, dont know why...
- rohit_singh4 years ago
Solution Sage
Hi Anonymous ,
You have to expand the table by clicking on the button highlighted below and select the EU LOC ID column