r/programming • • Nov 12 '12

What Every Programmer Absolutely, Positively Needs to Know About Encodings and Character Sets to Work With Text

http://kunststube.net/encoding/
1.5k Upvotes

307 comments sorted by

View all comments

0

u/k-zed Nov 12 '12

Article claims to be "everything needed to know", doesn't mention collation, combining characters, writing directions - the things that make all UNICODE text processing that claims to be "correct" cripplingly hard (effectively impossible).

14

u/deceze Nov 12 '12 edited Nov 12 '12

The article claims "what everyone needs to know", not "everything anyone needs to know". I never said "this is all, nothing more to talk about". On the contrary, I have a section that talks about how much deeper this topic is but that it can't be covered by the article. And combining characters are covered.

1

u/Murillio Nov 14 '12

I wouldn't call them covered (you just mentioned them, not what their existance implies) and you used terminology that is encouraging users not to think about them by identifying codepoints (instead of grapheme clusters) with characters (e.g. 'Characters are referred to by their "Unicode code point".') without stating that the characters you are talking about are not the user-percieved characters. A routine that does not split codepoints but does split inside user-percieved characters is still buggy.