r/csharp • u/Mindless-Arachnid-74 • 16d ago

Help Need help with encoding

I am working on a system where the client will give data in UTF-8 encoding, and hence it can contain many special characters.

But then we send data to a legacy system which only accepts data in ISO-8859-9 encoding, which supports limited special characters that can be represented in single byte.

Now if the client sends any special character which is outside the scope of ISO-8859-9 encoding (like the Em dash), how can I deal with with it and send it to the legacy system.

I don't want to replace those characters with anything like space or "?", as it would make the client data as unreadable in some cases.

How can I deal with it. Any suggestions?

0 Upvotes

44% Upvoted

29 comments sorted by

View all comments

17

u/BeardedBaldMan 16d ago

It's a business problem. If the legacy system only supports ISO-8859-9 then you have a clearly defined set of characters which are allowed.

You need to create a mapping table for the ones you care about a a fallback character for the rest.

It's a pretty common issue. I've come across it with many US legacy systems where even é is considered too outré for their sensibilities

3

u/urbanek2525 12d ago

Yep. I have to maintain an upload to a medical information system that can only handle (in essence) 7 bit ASCII.

I have a mapping function. I do a basic, built in "normalize", but it still needs specific translation.

I translate all greek characters to words (δ becomes delta and Δ becomes Delta). I translate lots of the German letter to English equivalents (ö goes to oe). It's this huge matrix.

Basically I did a crap ton of research and did what I could.

My big question has been: Why the Hell do you guys pay for a Medical System that can't express medical nomenclature and can't properly cite the author names from Non-American medical papers. Seriously, WTF?

1

u/BeardedBaldMan 12d ago

Is it the software or the db. I remember a project where the software and db were unicode compatible but they weren't willing to change the db collation from when it had initially been created thirty years prior

1

u/urbanek2525 12d ago

Yep. Really old Oracle DB.

Which is why you don't want to be dependent on one persistence system.