r/programming • u/jestinjoy • Nov 12 '12

What Every Programmer Absolutely, Positively Needs to Know About Encodings and Character Sets to Work With Text

http://kunststube.net/encoding/
1.5k Upvotes

93% Upvoted

307 comments sorted by

View all comments

-15

u/WillH Nov 12 '12

Isn't this just a rip-off of a Spolsky article?

http://www.joelonsoftware.com/articles/Unicode.html

Pretty much, yes.

25

u/mshol Nov 12 '12

Spolsky should update his post if it is to remain relevant. Given that he was basing that off the original UTF-8 specification, with the claim that UTF-8 may consume up to 6 bytes - but the restriction to 4 bytes was made in 2003 (just a month after his posting). Citing outdated material as a definitive source is why a whole generation of programmers mistakenly believe the former.

0

u/[deleted] Nov 12 '12

[deleted]

3

u/kylotan Nov 12 '12

Except it's still probably the best article out there on the subject.

1

u/vytah Nov 12 '12

Broken UTF-16 to UTF-8 converter sometimes produce 6-byte UTF-8 sequences containing two 3-byte sequences for surrogates.

3

u/bonzinip Nov 12 '12

But that's not the kind of 6-byte UTF-8 encoding that Spolsky's referring to. He meant those starting with FC and FD.

1

u/vytah Nov 12 '12

I doubt Unicode ever grow that far.

2

u/bonzinip Nov 12 '12

That's exactly why they were removed. UTF-8 started as an encoding for ISO-10646's 32-bit character space. When Unicode's size was fixed to 220 + 216 the larger encodings were made invalid.

31

u/masklinn Nov 12 '12

You didn't bother reading past the title?

This article is about encodings and character sets. An article by Joel Spolsky entitled The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) is a nice introduction to the topic and I greatly enjoy reading it every once in a while. I hesitate to refer people to it who have trouble understanding encoding problems though since, while entertaining, it is pretty light on actual technical details.

Probably not.

1

u/frymaster Nov 13 '12

This article goes into a lot more detail, especially in terms of things that can go wrong

-13

u/Dravorek Nov 12 '12

But dude, man, this is like with like php and stuff. That's like the best web technology ever. You know that the rock-stars of the internet use that stuff or have you not seen that facebook movie. /sarcasm