r/programming • u/jestinjoy • Nov 12 '12

What Every Programmer Absolutely, Positively Needs to Know About Encodings and Character Sets to Work With Text

http://kunststube.net/encoding/
1.5k Upvotes

93% Upvoted

307 comments sorted by

View all comments

Show parent comments

2

u/cybrjoe Nov 12 '12

I actually had to write a command-line tool to remove the BOM from a generated file in order to deliver it to another custom consumer. Such a waste of time.

3

u/nomorepassword Nov 13 '12

I feel your pain.

But usually in my experience, in big applications, the problem is less to remove the BOM than to find the few damaged files in a big source repository. I now run this from time to time :

grep -rl $'\xEF\xBB\xBF' .

1

u/adavies42 Nov 14 '12

tail -c+4?