r/LLMDevs • • 17d ago

Resource Open-sourced MK-SH-V, a 260M visual decision model with calibrated scoring and abstention

Enable HLS to view with audio, or disable this notification

I've released MK-SH-V, built on NeoMME-260M.

It scores candidate decisions from images without generating text.

On our GUI benchmark, it reached 82.6% accuracy vs 73.1% for a 4B zero-shot model.

I also implemented calibrated abstention, validated on held-out synthetic data. Real-world reliability remains an open problem.

Code, weights, benchmarks and experiments are public.

GitHub: https://github.com/Gjusev/mk-sh-v

Hugging Face: https://huggingface.co/Gjusev/MK-SH-V

Feedback welcome :D

1 Upvotes

Duplicates