Forum Discussion
convertation from utf-8 to utf-16
Dear colleagues!
I am getting string data from ODBC in UTF-8 encoding. Can anybody explain how to convert this data to UTF-16 using Power BI functions?
Something like it's possible in .NET:
Private Function Utf8ToUtf16(utf8String As String) As String
' Get UTF-8 bytes by reading each byte with ANSI encoding
Dim utf8Bytes As Byte() = Encoding.Default.GetBytes(utf8String)
' Convert UTF-8 bytes to UTF-16 bytes
Dim utf16Bytes As Byte() = Encoding.Convert(Encoding.UTF8, Encoding.Unicode, utf8Bytes)
' Return UTF-16 bytes as UTF-16 string
Utf8ToUtf16 = Encoding.Unicode.GetString(utf16Bytes)
End Function
5 Replies
Hi Anonymous
You could use M function
TextEncoding.Utf8 or TextEncoding.Utf16
Best Regards
MaggieCommunity Support Team _ Maggie Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.Hi v-juanli-msft,
Thank you for information. I have tried to use this functions like this:
#"Added Custom" = Table.AddColumn(Source, "CUSTCODEUTF16", each Text.FromBinary(Text.ToBinary([CUSTCODE], TextEncoding.Utf8), TextEncoding.Utf16))
similar I did it in .Net
But unfortunately get wrong result:
11\Р РћР—РќР˜Р¦Рђ
was converted to: ㄱ큜슠킠토킛钀ꃐ鳑ꃐ飂ꃐꛂꃐ鋑
It's look that I am doing something wrong.
Can you send some example - how to use this functions for conversation?
Hi Anonymous
As searched , the difference between utf-8 and utf-16 is how many bytes it require to represent a character in memory. UTF-8 uses minimum one byte, while UTF-16 uses minimum 2 bytes.
When you import data into Power BI, Power BI will cast the data to the corresponding data type.
Data type in Power Query are listed as below:
For example, Text type - A Unicode character data string. Can be strings, numbers, or dates represented in a text format. Maximum string length is 268,435,456 Unicode characters (256 mega characters) or 536,870,912 bytes.
So you don't worry about the filed memory.
Best Regards
MaggieCommunity Support Team _ Maggie Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
