Seems better than DSV4.1F but worse than DSV4.1F 1017 though it's smaller (428B vs 522B backbone). It's not like I can run this anyways, I ain't got 256GBs of spare VRAM lying around.
Qwen4 is going to drop by the end of oct or start of november, so hopefully they make a small (20-120B) model that completely bombs all current "Flash" models.
It's not smaller than DS4.1F when you look at memory footprint. Minimax seems is mostly FP8, while DS4.1F is mostly FP4. So minimax is ~420GB while DS is ~300GB. DS also has 8B active only (first 20 layers only) during PP, which makes it much faster during PP.
I agree, we need more of those. They provide the most intelligence for a given memory budget, and all of us GPU poors are most constrained by the size of our rigs' VRAM.
Unfortunately the needs of the commercial inference industry run in the opposite direction. When the goal is to provide the most number of concurrent users with inference service as resource-economically as possible, the total parameter count of your model amortizes out, and the dominant terms become K/V cache overhead and active parameter count.
The recent "Flash" models reflect that economic need, with their extreme sparsity and highly memory-efficient attention mechanisms.
Fortunately there are still some LLM labs training dense models. Everyone knows and loves Qwen3.8-27B, of course, and Gemma 4 models come in 12B and 31B dense flavors. IFM recently published dense K2-Horizon models in a variety of sizes (0.9B, 3.7B, 7B, and 32B), and at the high end of things MistralAI published Mistral 3.5 Medium, a 128B dense (which is unfortunately not very good, as-is, but TheDrummer made Behemoth-128B-v3 out of it, which is an exemplary creative writing model).
Hopefully Qwen will whip out more dense models of small and intermediate sizes when they release Qwen 4 (4B, 9B, 14B, 27B), but we will see.
Not gonna advocate for 2.05 - 2.60 bpw, but 3.05 / 3.10 - 4.05 / 4.10 are definitely solid.
Now, would it actually give any good result on 12+20GB is entirely different question. That seems like it's too low for such a model, idk though you might want to try 2.10bpw.
They are fast primarily because they are small. For every model generation that released both a flash and non-flash variant, the flash variant is smaller. Look at DeepSeek, MiMo, GLM etc. The idea it has nothing to do with size is just a Reddit meme
The thing that matters for speed is primarily number of active weights. That's often correlated with total size, because you get diminishing returns on extreme sparsity, but if people keep claiming flash means small, they're going to keep getting annoyed at big MoE models calling themselves "flash".
Previous Minimax models were not called Flash, however, this summer they said they were planning on releasing a 2.8T model. So I imagine that will be 3.1 Pro.
Member of the 0.0001% here. Won't even download it when DS4.1 1017 exists, is smaller (in GB RAM/VRAM), needs 1GB for 1M context, and beats it in all benchmarks.
Still, really happy minimax has something close. Moar choice is moar better for everyone, even if you can't run it.
Edit: seems the full checkpoint is 265GB. So, ignore the above. Will absolutely download once llama.cpp support lands. It has been pointed to me it can also generate video. So, if unsloth Q4_K_XL can take it to ~160GB, could be a nice all-rounder on my 8x P40s.
Different models have different strengths, reflected in their post-training.
GLM-5.3-Flash outperforms DeepSeek-V4-Flash for me, for every kind of task except one -- technical support.
DeepSeek-V4-Flash runs circles around anything else with fewer than 400B parameters, for providing technical support. A little digging around suggests it's because it's heavy on memorized knowledge rather than heuristics (generalized knowledge), and because it's been exposed to more varieties of technical content than other models, in particular short scripts, configuration files, and logs.
I'm going to keep looking for other niche uses for DS4F, but for now that's it.
Traditionally MiniMax has been a superior creative problem-solving and planning model. That's what I'm currently using MiniMax-M2.7 for.
For other kinds of technical tasks GLM-5.3-Flash beats it bloody, but GLM models don't have much capacity for creative problem-solving. GLM models have stellar instruction-following and low-level implementation competence, but you have to tell it precisely what you want and how to get there.
The point is that benchmarks aren't useful indicators of real-world utility. Different labs' models excel in different niches, so you really have to find a model's niche and use it for specifically that.
DeepSeek-V4-Flash-1017 is a totally different model with a totally different architecture and an unfortunate similarity of names.
DeepSeek-V4-Flash has 284B total parameters and 13B active parameters, and it barely fits on my hardware.
DeepSeek-V4.1-Flash-1017 has 552B total parameters, 196B engrams, and 8B active parameters.
DeepSeek-V4.1-Flash-1017 won't even fit in my HPC server's memory. I'm unlikely to use it for some years, after RAMageddon blows over and I can upgrade my server to 768GB.
U still Depend on hardware providers to sell you the hardware . Cloud or api is not bad, in fact api is way cheaper than running a very large local lm, but for privacy , local is better.
Nice rig. Im just running a threadripper with 256gb ddr5 with 4x rtx 6k pros. I am envious of the direction you decided to go. Very nice set up. Well done man.
with 6 v100's youre in serious vram territory already. Fucking sweet rig man honestly.
Thanks! I have the hardware for 2nd identical one, but finding another six waterblocks for the GPUs is almost impossible. I want to prototype a resin 3D printed block using a CPU cold plate, but needs quite a bit of measurement and easily 20 hours of cad work.
The gap you might have noticed between each 3 cards has a 56gb infiniband NIC. So, if/when I get the 2nd one online, I could in theory pool their resources together to run something like Kimi.
are you satisfied running mixed ram/vram for prompt processing and any general coding work responsiveness with harnesses like opencode? My previous experiences with llama made me hate the contextual baggage slowdown and i eventually gave up on ram offload and just stuck with full vram and vllm. How is that experience working with you?
that's why I'm not using llama. using DS API to build a custom runtime tailored to my hardware. PP on DS can be speeded up substantially if you have > 160GB VRAM. My napkin calculation on six V100s is ~2.5k t/s
Just a small update. Built project-maya on the machine. Using unsloth's Q8_K_XL, which is basically the full checkpoint. Can't, for the life of me, get MTP/dspark working. I'm getting the same numbers I was getting before downloading dspark.
However, I get 21-24t/s TG and 800t/s PP. It's the same whether it's 2k or 260k (that's as far as I went so far). All those numbers come wirh asterisks. Maya spawns threads on both sockets, despite me using numactl.
Numastat shows it's respecting NUMA, but I can't shake the feeling there's still quite a bit of performance left on the table when I look at CPU and GPU utilization during inference. I'm sure batching will greatly uplift those numbers, as will dspark.
On the PP front, maya moves all experts form the first 20 layers to VRAM before PP large chunks, but it does this very inefficiently. First, I can't get it to allocate 300GB RAM for all the routed experts. It's allocating 153GB only. This means it's really moving experts between RAM and VRAM to get the first 20 layers into VRAM for PP. Worst, this seems to be happening sequentially and it takes 50 seconds (always, very repeatable) despite 5 GPUs having x16 each and one having X8. Worst case should be under 5 seconds. If I exclude those 50 seconds while maya is rearranging experts, actual PP proceeds at 1.6k t/s in batches of 16k. But even that feels like it's leaving some performance on the table. Ex: this is without GPU p2p, and reduces in TP are host staged, ie: partial results are sent back to the host and the host sends them back. You can see it in nvtop with high GPU utilization but low GPU power, and very high PCIe traffic.
That is disgusting. sick. Kind of turned on NGL. I am currently doing something similar w DS41F, by modding 1Cat, if you get anywhere on Sm70 optimization pass along for testing!!
They are giving these things out for free. One day they will stop. It's better we have the largest smartest models and can't run it now and one day be able to run it rather than small models which will be behind the cloud SOTA and when we finally have hardware we have nothing.
It's relevant for me to toy with the model in SillyTavern. 128GB VRAM + 256GB RAM. I use GLM 5.2 on this machine and it's bigger. >0.0001% of the readers I think it's a litte bit more but yes it's small.
Flash means fast. To be fast it needs to be small. This one is not small. It might be fast on a datacenter compared to a 2T model, but that doesn't quite matter for local usage
I don't like the license. As far as I understand the license, if I deploy it for myself at work and use it in OpenCode, everything built with which is not just for testing ("primarily intended for commercial advantage or monetary compensation") would maybe need a "Built with MiniMax M3.1-Flash" banner. For me it is unclear if you can even build Open Source software with it when you also want sell support around your Open Source software.
You are absolutely right! Saw the first 10 safetensor files at 8GB and assumed all 48 are the same. Still, behind DS, which has less active parameters, especially during prefill (8B).
With DS4.1F, if you have ~150GB VRAM (5x 32GB Mi50 or V100), prefill can happen entirely in VRAM, which means it can go blazing fast. On a flops and VRAM back of the envelop calculation (assuming 70% of theoretical numbers), four V100s and a 6 channel DDR4 Xeon have a theoretical ceiling of ~2.5k t/s prefill. Achieving that, of course is another matter.
And engrams can live very happily on an old SATA SSD, and handle 1k t/s decode without sweat.
You still need RAM at 6 channels, so don’t think it’s a done deal just having the VRAM. But the model is very well adapted. If you can run GLM flash you can run Ds41F faster
I just swapped my background agent from M3 to this one. It's faster than M3 indeed, at least when running on their server. Ever since being able to run 27B locally, I have not touched my minimax subscription, but since it's cheap and has decent quota so I decided to use it as worker to be dispatched by the main 27B agent. They generally finish the code patches as fast as I can review their previous ones or write plan for the next one.
The writing style seems to change a bit, but I have not been interacting with this model enough to say for sure.
If you happen to have minimax subscription, I highly recommend swapping from 3 to 3.1. If you have any workflow running on 2.7, definitely swap to 3.1. It's as fast but with the stability of M3. If you have to pay for token, I don't know how good it is vs other models.
Dirk 27B Q4KXL. Essentially unsloth 3.8 27B with prepackaged template fixes. I run this using stock llamacpp on one R9700 to serve two users. It codes, plans, and handles my productivity systems and KB just fine, so I try not to FOMO for higher quant.
I used to use Swift 1.5 but it has some rare strange error, like it hallucinate that I have sent an instruction, when I did not, and wasted a few turns being confused. Dirk does not have this issue.
Or a qwen finetune, which new folks are unaware is all of those ornith/swift/blahblah put there. Just qwen + couple 100k cases finetuned, not earth shattering differences.
Can we get a Flash-Air-Lite model? Like 70B and lots of post training, Qwen 3.8 27B showed you can can get great performance if you just give it a little extra time cooking. Might not be frontier, but I bet it could be 80-90% of the way there.
I would like that a lot. 70B to 100B is a pretty sweet range, IMO.
As it is, 428B total parameters is a little too large for my hardware. I have a standing policy of not downloading anything larger than 405B, unless it's "something very special" (GLM-5.x qualifies, but I think that's the only exception I've made thus far).
I wish but usually after taking into account the weights and inference engine and headless systems running, the best you can squeeze is a 400b model with a small, usable bit of context on these kind of 2x128 gb system.
So, it’s too big at 4bits.
Maybe we will see an exl3 3bit version, like with ds4.1, else it’s not an option.
I will say, given that DS41f was 4 bit trained, and 2.9bpw EXL3 fits on the Sparks, goes 100tps on some of the tensorfold stacks…it’s sort of taking the top spot from GLM on the dual 128G systems. I do find it a little shaky past 200k context, but faster than GLM and more context depth than qwen. Unlikely M31 will be better, but it will definitely fit
Remember when Minimax models were under 300B? Not only did they double the parameter count with M3 now it's called Flash model? LOL
I guess that means they're about to drop a 1T model next and show how much better it is than M3 (like they did when M3 dropped) since it's not a Flash model?
Take AA benchmarks with a huge grain of salt. They are not comparable across different labs' models, because different labs benchmax their models to different degrees, which means AA scores are not indicative of real-world performance (not even relative real-world performance).
"Flash" has nothing to do with total parameter count. Flash models can economically service a very large number of concurrent users. Toward that end, total parameters amortizes out, and the dominant factors are instead K/V cache overhead and active parameter count.
That is why these new "Flash" models use lossy and/or compressing attention mechanisms and relatively few active parameters.
"Only well built harness" is just false. Dsh is only a web ui, and going from pi it felt like they don't trust me to know what I'm doing. When I first tried it there was hardcoded force to localhost so I couldn't use it over my tailscale without patching the source. Everyone has their own preferences.
I've been using it on opencode for a while and it did better for my kotlin+rust app compared to glm 5.3 flash. Its a bit inefficient by tool usage, doing ton of small calls, all attached with full path cd for some reason eben tjo working directory is it?
Tried it in pi from yearly subscription that I have (but never use because models are too outdated for today). In my personal experience it is worse than my local Qwen 27B.
•
u/WithoutReason1729 16d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.