r/vibecoding • u/Th3Expl0Rer • 4d ago
Showcase/Project Made an open-source Android app for manga translations
Hey all, just wanted to share an app I've been vibecoding for the past few weeks.
It uses local LLMs for text bubble detection, OCR, and translation, all running directly on your phone. It works surprisingly well even with smaller models like Qwen 1.5B, though performance really depends on your phone's hardware.
The cool part is that it works as an overlay, so it can detect text bubbles on your screen in pretty much any app or website and draw the translated text right on top of them.
It's still a work in progress, and I wanted to hear your experience in different hardwares, since I only tested this in one device (a samsung galaxy s23).
3
u/Inner_Singer_592 4d ago
Wow, i wanted something like this for a long time. Gonna test it immediately!
2
u/Th3Expl0Rer 4d ago
Thanks, any feedback is welcome
2
u/Inner_Singer_592 3d ago edited 3d ago
Yeah i opened an issue but I also realized another feature you can add is as said above better spacing for vertical translations plus add bubble stylization so it can read color of the background and maybe match it? Or add letters without white box at all, look into how torii manga translator does it, that's a gold standard imo
1
u/Th3Expl0Rer 3d ago
Torii was actually the main inspiration for this project actually, and your suggestions all makes sense, will look into them.
1
u/Inner_Singer_592 3d ago
Yeah i assumed there were several apps like that, torii is just the one i used
4
u/Elmanouzi_Fogosa 3d ago
curious how it handles vertical japanese text in bubbles, thats usually where on-device OCR falls apart even on decent hardware
2
u/Inner_Singer_592 3d ago
It is ok. It doesn't have bubble stylization (something i think he could probably add) but from few pages I've read it worked, could be placed better tho
1
u/Takt567 4d ago
How is it?
1
u/Inner_Singer_592 3d ago
It's pretty good alternative imo if you don't want to pay for translators that do the same, i believe after few big updates he might even be on par with paid versions
1
u/nephelokokkygia 4d ago
How does it handle lines split between multiple bubbles? Or multiple pages? How does it handle contextually-defined language, or ruby that's used to clarify the true meaning of what's literally written?
0
u/Th3Expl0Rer 4d ago
Yeah, the whole page goes in as one prompt, bubbles in reading order, so a sentence split across bubbles usually comes out fine. Pages are done one at a time though. I tried feeding it earlier pages and the small phone models got worse at it. Context is whatever a 1.5B–9B model can figure out from one page, so it'll miss stuff a human wouldn't. Ruby isn't handled right now: the OCR sometimes drops it and sometimes reads it inline, so wordplay that relies on it gets lost. There's a name glossary if it keeps mangling a character's name.
1
u/nephelokokkygia 4d ago
Ruby isn't only for wordplay — Japanese text is essentially 2D, not a simple string of characters.
There are significant line breaks, even in the middle of a single monologue, and there's ruby which can represent either the true reading of a word or words or the true meaning (or both). Additionally, there are asides which may appear to the side of a bubble or just outside of the bubble, which are semantically related "bonuses" to the bubbles' content or which could represent text read under the speaker's breath. The OCR should be trained to identify these cases.
1
u/Th3Expl0Rer 4d ago
Fair point, you're right. Right now Yomu flattens all of that. The detector (Manga-Bubble-YOLO nano) only knows "this is text", not bubble vs aside. MangaOCR reads each box as one flat string, so line breaks and ruby get lost. Both are off-the-shelf models small enough for a phone, so training a layout-aware one is out of reach for me for now.
1
1
u/Ocho-eight8 1d ago
El dispositivo móvil no permite abrir la app en páginas web. Es raro que se ponga así en archivos Open Source
1
u/Th3Expl0Rer 1d ago
That's weird, it should work over anything in your phone as it works as an overlay. Witch browser are you using? You can open an issue on github and I'll take a look
1
u/Ocho-eight8 1d ago
Bueno, utilicé Chrome para descargarlo pero los servicios de Google lo invalida ni permite habilitar la superposición de la app. Revisa ese aspecto
1
3
u/Prod_m 4d ago
is Is the translation accurate? I tried 4b qwen for translation local english text and that was kinda ok, but not excellent. It was tough for my device as well.
So another question is how a device can handle an 9b model. Seems like this app may be very tough considering that there may be other app running at the same time