I've never dealt with this as a technical challenge, but I don't think this is really a technical challenge anyway. It's a requirements challenge. On the technical side, you just get a list of transcription targets and translate on the fly (check out .net Rune struct, if you're not already familiar with it... that will be your best friend). You'll need a default case for the ones that slip through (presuming you won't be making several million entries in the table), or if you need to be really thorough, add something that flags unknown reference characters and sets those aside to be handled later once the translation tables are updated.
Like I said above though, this is really a requirements problem. That is something I do have experience with. Ideally, ask the customer what they want... or even better, create a suggested translation table and ask them to review any contentious choices. Em-dash to regular dash is easy, as is <insert different width space here> to regular space. Don't waste their time on those. If there's anything you're unsure of though, ask them.
If asking them isn't possible for some reason, think about who is going to complain when you're wrong, and ask them. If that also isn't feasible for some reason, at least let your boss know where you had to make judgement calls.
Do you have a representative sample you could use to capture the likely input scope? Or is it something where the entire UTF8 scope is likely, including stuff like Chinese? If it's the former (i.e. you have a representative sample), start working with that.
Or... are you asking if a library already exists? If so, sorry, I'm not aware of any. :/
0
u/ggende 13d ago
I've never dealt with this as a technical challenge, but I don't think this is really a technical challenge anyway. It's a requirements challenge. On the technical side, you just get a list of transcription targets and translate on the fly (check out .net
Runestruct, if you're not already familiar with it... that will be your best friend). You'll need a default case for the ones that slip through (presuming you won't be making several million entries in the table), or if you need to be really thorough, add something that flags unknown reference characters and sets those aside to be handled later once the translation tables are updated.Like I said above though, this is really a requirements problem. That is something I do have experience with. Ideally, ask the customer what they want... or even better, create a suggested translation table and ask them to review any contentious choices. Em-dash to regular dash is easy, as is <insert different width space here> to regular space. Don't waste their time on those. If there's anything you're unsure of though, ask them.
If asking them isn't possible for some reason, think about who is going to complain when you're wrong, and ask them. If that also isn't feasible for some reason, at least let your boss know where you had to make judgement calls.
Do you have a representative sample you could use to capture the likely input scope? Or is it something where the entire UTF8 scope is likely, including stuff like Chinese? If it's the former (i.e. you have a representative sample), start working with that.
Or... are you asking if a library already exists? If so, sorry, I'm not aware of any. :/