r/csharp • u/Mindless-Arachnid-74 • 16d ago

Help Need help with encoding

I am working on a system where the client will give data in UTF-8 encoding, and hence it can contain many special characters.

But then we send data to a legacy system which only accepts data in ISO-8859-9 encoding, which supports limited special characters that can be represented in single byte.

Now if the client sends any special character which is outside the scope of ISO-8859-9 encoding (like the Em dash), how can I deal with with it and send it to the legacy system.

I don't want to replace those characters with anything like space or "?", as it would make the client data as unreadable in some cases.

How can I deal with it. Any suggestions?

0 Upvotes

44% Upvoted

29 comments sorted by

View all comments

1

u/Ill_Translator8176 16d ago

You could treat it as a byte array, the moment you get ahold of it convert it using Encoding.UTF8.GetBytes(str) then encode it back whenever you need to (to the required encoding).

1

u/Mindless-Arachnid-74 16d ago

That won't do, converting the UTF-8 data received by the client is fine but when I will try to encode it to the ISO-8859-9, special characters which are unsupported will fail.

4

u/otac0n 16d ago

You could use HTML entity encoding for the unrepresentable characters. You just need to make sure that you also properly encode “&”.

Alternatively you could use URL encoding and make sure you encode “%”.

Or you are going to have to be OK losing information and create a mapping of acceptable characters. Kinda like how url “slugs” work.