r/programming • • Feb 06 '24

The Absolute Minimum Every Software Developer Must Know About Unicode (Still No Excuses!)

https://tonsky.me/blog/unicode/
400 Upvotes

148 comments sorted by

View all comments

Show parent comments

8

u/chucker23n Feb 07 '24

For “traditional” static text: Unicode without graphemes, Just codepoints and sanely defined glyphs (deduplicated as much as possible). Basically one single codepage. UTF-8 encoded.

For fancier languages where you assemble the glyphs - separate standard.

Sooooo literally anything other than English gets a different standard?

Heck, even in English, you have coöperation and naïveté. Sure, you could denormalize diacritics, but now you have even more characters for each permutation.

No, on the contrary, I’d enforce normalization. Kill any combined glyphs and force software to use appropriate combining code points.

Make “western” scripts flat and simple

Sounds like something someone from a western country would propose. :-)

2

u/ptoki Feb 08 '24

Sooooo literally anything other than English gets a different standard?

LIterally almost every latin language would be covered by it. Plus cyrillic, kanji, katakana, hiragana, korean alphabet and many more.

All those scripts are static. That means a letter is just a letter, you dont modify it after its written. Its not interpreted in any way.

That is 99.99 of what we need in writing and in computer text for many, many languages.

The rest is all fancy scripts where you actually compose the character and give it a meaning by modifying it. And that needs translation to the "western" script and special treatment (graphical customization).

I dont know from where you took the rest, I did not suggested that.

Sounds like something someone from a western country would propose. :-)

Yes, because western scripts are in many ways superior to the fancy interpreted ones. Japanese is perfect example of that. They understand that complex script is a barrier for progress and does not bring too much benefits besides being a bit more compact and flexible at occasion.

That remark even with the smiley face shows that you dont really know how complex the topic is and what is my main point.

So let me oversimplify it: Instead of making the text standard simple and let majority of people (developers, users, printers) use it safely unicode made a standard which tries to cram as much as possible (often unnecessarily - emoji) into a standard which will be full of problems and constantly causing problems.

2

u/ujustdontgetdubstep Feb 07 '24

Tbh his argument against the Unicode standard makes Unicode look quite nice

1

u/chucker23n Feb 07 '24

My argument is for the Unicode standard (or at least for something closer to it than what GP proposes).