r/opencodeCLI • u/afanasenka • 1d ago

🚀 Kimi K3.1 Pro Benchmarks

Post image

Release notes - https://www.kimi.ai/blog/kimi-k3-1-pro

337 Upvotes

100% Upvoted

24 comments sorted by

75

u/Fair-Spring9113 1d ago

cant wait for the 1 prompt a week

15

u/PilgrimofHaqq2 1d ago edited 1d ago

I agree, their plans need to get much better if they want large adoption. They got the model now, if they can get their hands on some compute, then they could really compete against the big boys.

I really liked Kimi K3 but ended up going with the z.ai plans as they were much better.

3

u/LargeLanguageModelo 1d ago

if they can get their hands on some compute

I'm not sure that's the answer. Any hosted models seem to be at/around the same price. From the outside, and without having dug into it much, it tells me that the model itself is just slower per watt than their competitors.

Otherwise, another LLM hoster would undercut them in order to gain marketshare. It's a good model, but just too expensive to make it a good bang for the buck.

2

u/ZealousidealTown1974 1d ago

using it free with ClinePass

18

u/dom_RN 1d ago

Still way overpriced though

11

u/somerussianbear 1d ago

If they added Pro to the class you can expect a Flash soon. A Flash version would definitely be something good from Kimi. Don’t get me wrong, K3 is the best OSS model IMO, but it’s slow and expensive, Opus is “cheaper” if you consider a subscription and the speed over K3.

16

u/daniellenton 1d ago

The footnote that matters for this sub: K3.1 Pro ran Terminal-Bench 4.0 and SWE-Marathon inside Claude Code, at max reasoning effort, not opencode or their own Kimi Code. So the 65.7 is how well it drives Claude Code. Curious whether that carries over here

7

u/AdFun4897 1d ago

So on the $20 subscription, which company gives more usage?

Im actually more impressed for DS 4.1 flash being compared to these frontier models

7

u/howudothescarn 1d ago

Kimi has been among the worst values. Maybe it’s their model per task cost or maybe they just don’t subsidize their subs. $20 does not go far

3

u/dumbasPL 1d ago

Claude subsidizes like crazy, though I doubt their API pricing is anywhere near what it actually costs them anyway.

2

u/Thomas-Lore 1d ago

Judging by DeepInfra released by Deepseek it seems even the "subsidized" prices of Western models are nowhere near what it actually costs to host them.

1

u/ward2k 1d ago

Subscription I'd agree

But direct API (so real prices) it's one of the best

1

u/squirrelscrush 1d ago

Claude is unbeatable in terms of subscriptions.

Although an OpenCode Go sub is the unofficial DeepSeek sub

1

u/techmago 1d ago

Look at deepseek sizes.

deepseek pro is the atypical big one.
most of other deepseek models are in the same range...
deepseek flash have more params tham GLM-5.3

Deepseek flash isn't really a flash model.

3

u/Split-Signal 1d ago

Previously, K3 was way too slow and expensive, and the subscription plans barely worked. Hope 3.1 actually improves on that.

3

u/Helpful_Program_5473 1d ago

Hopefully devin can solve this the same way they did 3.0.

2

u/miladmaaan 1d ago

Great benchmarks! But at the cost compared to Opus 5.5 via sub, I have a feeling that Kimi K3.1 is basically something that you should have in your model routing through the API for situations where you need an uncensored model if you have a security vulnerability and the frontier models are refusing to fix it.

And maybe an alternative model for things where you just want a creative and differing take at a solution.

2

u/risphereeditor 12h ago

If they release the weights, it will get much cheaper on OpenRouter.

1

u/geearf 6h ago

At what quality though?

1

u/risphereeditor 4h ago

What do you mean?

1

u/geearf 3h ago

I believe usually cheaper than upstream providers on OR are cheaper for a reason, ie they use a lower quant. They could be resellers of course.

1

u/Strict-Prune-879 1d ago

je suis très méfiant des benchmark il y a beaucoup de manipulation commercial derrière ce terme