A Simple Method to Filter Text Strings in Query Editor

[GregBeaumont]

Using M in Query Editor, I'm looking for a simple and efficient means by which to remove non-letters from a text column in a table. For example, if the first two rows within a column have values "this is not all_text!" and "1234Letters$" I'd like them to become "this is not all text" and "Letters". A few entries also have emojis that are pulled into the query, and I'd also like to scrub those out. Any suggestions are appreciated!

[ImkeF]

I'm not aware of a "non-letter"-class in M.

Instead we have to define specificly a black- or whitelist: What shall stay in or what shall be removed?

Which route would you prefer?

[GregBeaumont]

I'm thinking I should set everything to lowercase, then create a whitelist for a-z. Probably the best way to eliminate odd characters. Thank you.

Greg B

[ImkeF · Accepted solution]

No need to lowercase. Try this:

Text.Remove(..YourText.., List.Transform({0..64, 91..96, 123..50000}, each Character.FromNumber(_))))

This will remove every character within the range of 0..50000 that isn't a..z or A..Z. So if you have some special signs that you want to include, you need to find the number (Character.ToNumber) and include those numbers in the list above.

[pfuschi]

Hi Imke

This also removes characters written in non-latin alphabetics....

Any suggestions?

[ImkeF]

You need to find the corresponding numbers to these characters. Easiest might be to create a " translation-table", load it to excel and then search & Identify your ranges: Create a list: {1..50000}, convert to table and add a column: Character.FromNumber([ColumnName]).

[Brian_M]

Nice!

[Greg_Deckler]

Well, Text.Remove provides a nice way of removing a list of characters but Text.Replace does not provide the functionality.

https://msdn.microsoft.com/en-us/library/mt260472.aspx