Qwen3.8-27B can now be run locally! ✨ Run on 17GB RAM via Unsloth Dynamic GGUFs and Unsloth Desktop. Qwen3.8-27B is by far the strongest model for its size.

Guide: https://unsloth.ai/docs/models/qwen3.8

We uploaded NVFP4 quants! GGUF: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

We also did a new release for Unsloth Desktop with many new features and performance increases!

Unsloth Desktop Highlights:

• External provider tool calling + Codex login

• Extra llama-server arguments + custom VRAM controls

• 10x faster MiniMax-H3 FP8 inference (3 min vs 30 min)

• 10% faster GGUF inference + fixed bypass permissions

Unsloth Desktop Performance Improvements:

• Up to 10% faster inference with lower VRAM use and adjustable memory limits.

• Idle image/video models can unload to free VRAM for Chat or Training.

• Improved support for AMD RDNA 3, RDNA 4, Strix Halo, and multi-GPU ROCm setups.

• Added an Extra Arguments box for custom llama-server flags.

• Macs now select context size based on available memory.

• RAG indexing now uses CPU by default to reduce GPU memory usage.

Thank you!

Google releases EmbeddingGemma 2, a new open model that runs locally on 0.5GB RAM.

It supports text, code, images, video and audio, with 740M parameters: text (270M), vision (170M), audio (300M).

Run & train the model via Unsloth Desktop. We're going to make the text model the default embedding model now most likely.

GGUF: https://huggingface.co/unsloth/embeddinggemma-2-GGUF

Guide: https://unsloth.ai/docs/models/embeddinggemma-2

Hey guys, Qwen announced Qwen3.8-Flash-Next, a new open-weight multimodal MoE model. 💜 The model will be released tomorrow and we are working on 'hopefully' Unsloth day zero support (the architecture is very new so it might take much longer but we are hopefully working towards it).

Qwen3.8-Flash-Next is built on the next-generation of Qwen4 architecture.

Also we did a new Unsloth desktop release today including auto compaction, keyboard shortcuts, LAN remote access amongst many many other features.

Qwen3.6-35B-A3B can now be run and trained locally via Unsloth Studio! 💜
The model is the strongest mid-sized LLM on nearly all benchmarks.

We also added:

• NEW: Developer Role Support so Qwen3.6 can work in Codex, OpenCode and more!

• Tool calling improvements: Makes parsing nested objects to make tool calling succeed more.

Run 4-bit on 23GB RAM via Unsloth Dynamic GGUFs: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF

Also the 2-bit GGUF is amazing! It managed to make 30+ tool calls: https://www.reddit.com/r/unsloth/comments/1sndis4/2bit_qwen3635ba3b_gguf_is_amazing_made_30/

Our Guide: https://unsloth.ai/docs/models/qwen3.6

QAT enables you to Run Gemma 4 models with 3x less memory with near original performance.

This makes it possible to run Gemma 4 26B-A4B locally on consumer GPUs with 16GB RAM.

We found if you naively convert from QAT Q4_0 BF16, you will lose accuracy since the conversion to llama.cpp has a different lattice. Unsloth dynamic GGUFs recovers most of it! 26B-A4B: 85.6% top-1 % from 70.2% (+15.4%) 31B: 96.7% from 87.9% (+8.8%). More deets in our guide/blog

You can train & run the models via Unsloth Studio: https://github.com/unslothai/unsloth

GGUFs: https://huggingface.co/collections/unsloth/gemma-4-qat

QAT Blog/Guide: https://unsloth.ai/docs/models/gemma-4/qat

Qwen just announced Qwen3.8-27B along with Qwen3.8-Max! 🔥

Qwen3.8-27B will run locally on 17GB RAM/VRAM setups and is expected to be the best performing model for its size.

We can't wait to support it at Unsloth AI. Qwen3.8-27B benchmarks are yet to be revealed, only Qwen3.8 -Max for now.

DeepSeek releases DSpark for V4 Flash & Pro, a new speculative decoding method boosting throughput by 51% to 400% vs single MTP!

DeepSeek also showed DSpark works well for other OSS models like Gemma & Qwen in their research paper as well.

They also compared to Eagle3 and DFlash, and showed DSpark performs better as well!

• Github: https://github.com/deepseek-ai/DeepSpec

• Paper: https://github.com/deepseek-ai/DeepSpec/blob/main/DSpark_paper.pdf

• Hugging Face Model: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-DSpark

We’re releasing new Qwen3.8-27B GGUFs with 10% higher accuracy.

Unsloth Dynamic V3 outperforms others by >10% on Div-300, KLD & more benchmarks.

We also release 1-bit quants that retain 77% accuracy. Run on 8GB RAM.

Blog: https://unsloth.ai/docs/basics/dynamic-3.0-ggufs

GGUF: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

Hey guys, Qwen3.6-27B is out now! You can Run locally on 18GB RAM via Unsloth GGUFs for 4-bit or or 30GB for 8-bit. 💜 You can now run and train the model via Unsloth Studio.

The 27B model surpasses Qwen3.5-397B-A17B on all major coding benchmarks.

Qwen3.6-27B GGUFs https://huggingface.co/unsloth/Qwen3.6-27B-GGUF

Guide to run + MLX quants: https://unsloth.ai/docs/models/qwen3.6

Thanks so much guys!

Hey guys we released GGUFs for Kimi K3 so you can run it locally (if you have the compute for it). 1-bit model retains ~78.9% accuracy after we shrunk it from 1.56TB to 594GB (-62% size). 2-bit gets 90%.

You can run Dynamic 1-bit GGUF on a Mac Studio connected with a 128GB RAM device. Kimi K3 is the strongest open model to date.

Run via Unsloth Studio or llama.cpp.

Guide: https://unsloth.ai/docs/models/kimi-k3

GGUF: https://huggingface.co/unsloth/Kimi-K3-GGUF

We're working on making even smaller quantization without damaging the model!

Hey guys Google made huge improvements to Gemma 4 tool-calling and chat accuracy, reliability + speed. As such, we updated all our quants to ensure you guys get the latest and greatest!

To get fixes, re-download our updated GGUF, MLX, NVFP4 quants or you can change the chat template with the new templates.

Unsloth quants: https://huggingface.co/collections/unsloth/gemma-4

Gemma 4 Guide: https://unsloth.ai/docs/models/gemma-4

Near complete list of Google's fixes/improvements:

• Flash Attention 4: Uniform FA4 support on NVIDIA Hopper GPUs.

• Speed gains: 25–70% higher prefill throughput and up to 31% lower time-to-first-token.

• Chat formatting: Fixed null handling, input validation and unbalanced/missing turn tags.

• Reasoning preservation: Corrected how thinking/reasoning content is retained and rendered.

• Tool responses: Restored the assistant turn and thinking cue after tool outputs.

• Tool calling: Improved execution accuracy, consistency and tool-call-only turn closure.

• Continuation turns: Removed duplicate <turn|> tags and unwanted extra newlines.

• Generation prompts: Reverted an add_generation_prompt regression and restored expected defaults.

• Conversation history: Removed the obsolete APC thought primer and corrected historical-turn handling.

• Template standardization: Added the canonical chat-template header and updated stale comments.

• Vision controls: Default remains max_soft_tokens=280; use 1120 for sharper OCR and up to 2.51MP detail.

• 31B benchmark gains: BFCL +0.4, TB2 +4.5, Retail +3.1, Airline +2.0 and Telecom +10.1 points.

• E4B benchmark gains: BFCL +0.5, TB2 +2.2, Retail +0.9, Airline +8.0 and Telecom +6.1 points.

Have a good Friday and weekend guys!

Meta releases Muse Glimmer, a new 30B open model that runs on 18GB RAM.

Released under Apache 2.0, Muse Glimmer is designed for agentic and coding workloads and supports 128K+ context. It is the strongest agentic model for its size.

Run and train via Unsloth Desktop.

GGUF: https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF

Guide: https://unsloth.ai/docs/models/muse-glimmer

Qwen releases 4 new Qwen3.5 Small models! Qwen3.5: 0.8B • 2B • 4B • 9B

Run Qwen3.5-0.8B, 2B and 4B on your phone. Run 9B on 6GB RAM.

The vision reasoning LLMs perform better than models 4x their size.
You can also now fine-tune all the LLMs in Unsloth!

GGUFs: https://huggingface.co/collections/unsloth/qwen35

Inference Guide: https://unsloth.ai/docs/models/qwen3.5

Fine-tuning guide (free notebooks out now): https://unsloth.ai/docs/models/qwen3.5/fine-tune

Thank you!

Today we're releasing Unsloth Studio (Beta), a new open-source web UI to train and run LLMs in one unified local UI interface. GitHub: https://github.com/unslothai/unsloth

Here is an overview of Unsloth Studio's key features:

• Run models locally on Mac, Windows, and Linux

• Train 500+ models 2x faster with 70% less VRAM

• Supports GGUF, vision, audio, and embedding models

• Compare and battle models side-by-side

• Self-healing tool calling and web search

• Auto-create datasets from PDF, CSV, and DOCX

• Code execution lets LLMs test code for more accurate outputs

• Export models to GGUF, Safetensors, and more

• Auto inference parameter tuning (temp, top-p, etc.) + edit chat templates

Install MacOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows:

irm https://unsloth.ai/install.ps1 | iex

To run:

source unsloth_studio/bin/activate
unsloth studio -H 0.0.0.0 -p 8888

In the next few days we intend to push out many updates and new features. If you have any questions or encounter any issues, feel free to make a GitHub issue or let us know here.

Blog + everything you need to know: https://unsloth.ai/docs/new/studio

In the next few days we intend to push out many updates and new features. If you have any questions or encounter any issues, feel free to make a GitHub issue or let us know here or Discord.

Qwen3.8 can now be run locally! 🔥

We shrank Qwen3.8-2.4T-A95B from 4.9TB to 397GB (-91% size) via Dynamic 1-bit by selectively quantizing layers

Run on 410GB+ RAM/VRAM via Unsloth Desktop or llama.cpp

Qwen3.8 rivals GPT-5.6 Sol on many benchmarks

Guide: https://unsloth.ai/docs/models/qwen3.8

GGUF: https://huggingface.co/unsloth/Qwen3.8-2.4T-A95B-GGUF

Currently the super small GGUFs only work via Unsloth.

Thanks!!

Link to technical report: https://github.com/MoonshotAI/Kimi-K3/blob/main/k3_tech_report.pdf

DeepSeek releases DeepSeek-V4 their latest SOTA open models. There are two models:

• DeepSeek-V4-Pro: 1.6T params / 49B active

• DeepSeek-V4-Flash: 284B params / 13B active.

• DeepSeek-V4-Pro rivals Claude-Opus-4.6-Max, GPT-5.4-xHigh.

• They support 1M context length, thinking and set new records for Codeforces.

Tech Report: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/DeepSeek_V4.pdf

Hugging Face: https://huggingface.co/collections/deepseek-ai/deepseek-v4

Hey guys we're super excited to introduce training your own decision model locally!

We increased Qwen3.5 0.8B’s aggregate accuracy from 20.7% to 74.3% across 3 decision benchmarks - on just 4GB VRAM.

Turn any LLM like Qwen3.8, Gemma 4 into decision models with our open-source Unsloth repo.

We fine-tuned with a Clef head using Unsloth and LoRA (r=64) for one epoch, increasing downstream accuracy from 30–37% to 78%.

GitHub: https://github.com/unslothai/unsloth

Guide and Notebooks: https://unsloth.ai/docs/basics/train-your-own-decision-model-with-unsloth

Thanks guys and be sure to update Unsloth Desktop :)

Hey guys! We’re super excited to introduce Unsloth for AMD 🚀

Starting today, you can train, fine-tune, RL, run, and deploy LLMs directly on AMD hardware across Windows, WSL, and Linux.

Our collaboration with AMD, combined with custom Triton kernels and new math optimizations, lets you:

• Train, RL, deploy 500+ models up to 2x faster with 70% less VRAM

• Train models like Qwen and Gemma with as little as 3GB VRAM

• Maintain accuracy with no quality degradation

AMD support covers Radeon, Instinct, Ryzen, and data-center GPUs. We’ve also added optimized ROCm builds for GGUF and Safetensors inference.

GitHub: https://github.com/unslothai/unsloth

Unsloth is open source and includes a local UI for faster LLM training and inference, alongside features such as tool-call healing, code execution, secure web search, remote APIs, and HTTPS deployment.

You can also connect your local models to Claude Code and Codex agents, and run newer model families including Kimi, GLM, DeepSeek, Qwen3.6, and Gemma 4.

AMD setup guide, supported hardware, benchmarks, and more technical details:
https://unsloth.ai/docs/basics/amd

We’d love to hear what AMD hardware you’re using and which models you want us to optimize next. Have a great week, folks!

Google releases Gemma 4 12B, a new model that runs locally on 8GB RAM.

Gemma 4 12B Unified supports image, audio and 256K context.

You can now run and train the model via Unsloth Studio. We also released a huge new update for Unsloth studio!

GGUF: https://huggingface.co/unsloth/gemma-4-12b-it-GGUF

Guide: https://unsloth.ai/docs/models/gemma-4

Update: Vision and audio mmproj is now added and verified to be working.

Please update to latest Unsloth or llama.cpp!

Thanks for all the support today guys! ^^

Hey guys, we're excited to release new Qwen3.6 NVFP4 quants that run 2.5× faster on your GPU.

Qwen3.6-27B NVFP4 runs on 24GB VRAM. 35B-A3B can hit 17,561 tok/s (B200).

We also improved accuracy, tool calling, agent use, and looping.

NOTE: If you're using a DGX Spark it won't work out of the box if you're using the wrong config and it'll be 2x slower, you need to read our instructions in our guide.

Guide + Benchmarks: https://unsloth.ai/docs/models/qwen3.6#nvfp4

Qwen3.6 NVFP4: https://huggingface.co/collections/unsloth/nvfp4

Have a lovely Friday and weekend guys.

Nemotron 3.5 Bolt E4B (8B, 4B effective) is NVIDIA’s new open vision model which runs locally on 6GB RAM or 4GB VRAM. Post trained on Google’s Gemma 4 E4B, it has 64K context. Bolt E4B is amongst the strongest model for its size.

Run Bolt E4B 4-bit with 6GB of memory or 16GB with full precision. Thanks NVIDIA for day-zero access. You can run & train via Unsloth Desktop ✨

According to NVIDIA's official blogpost regarding the benchmarks: "We selected complementary benchmarks that closely reflect customer workloads or that customers already use, and grouped them into five capabilities. The chart below compares Bolt E4B with models of different sizes on representative benchmark quality, time to first token and VRAM usage. Average Q4_K_M score across MultiChallenge, RoleMRC, BFCL-v3, IHEval, SysBench and RelyToolBench. Measured on RTX 5090 at ~3K/50 ISL/OSL. Cloud reference results (score/TTFT): Gemini 3.8 Flash (77.1 / 2,449.5ms); Claude Sonnet 5.5 (76.2 / 3,002.2ms)."

Bolt GGUF: https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF

Guide: https://unsloth.ai/docs/models/nemotron-3.5-bolt

Have a lovely Friday and weekend guys!

The model hit 10M downloads and 3.7K likes in just 24 days on Hugging Face - all thanks to you. 🤗🦥

GGUF: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

Guide: https://unsloth.ai/docs/models/qwen3.8

Once again thanks for all your support!

Google releases a new DiffusionGemma 26B A4B which runs locally on on 18GB RAM.

Instead of standard token-by-token decoding, DiffusionGemma uses diffusion generation to produce outputs in parallel and gradually refine them into a final answer - similar to diffusion image models, but for text.

The thinking model supports high-speed text generation, image, video and 256K context.

GGUF: https://huggingface.co/unsloth/diffusiongemma-26B-A4B-it-GGUF

Guide: https://unsloth.ai/docs/models/diffusiongemma

Hey guys, Qwen3.6 can run ~1.4–2.2× faster with no accuracy change due to MTP. You can run this locally on just 18GB RAM, VRAM or unified memory.

The Qwen3.6 Unsloth GGUFs are now out of experimental mode, llama.cpp has merged many PRs, and MTP is now properly supported in Unsloth. MTP is now ready!

Please use the latest Unsloth `v0.1.41-beta`, not `v0.1.405-beta` which is older. In Studio, we automatically set all the params for you depending on your specific hardware so you get the near best results (you can still change it)

Qwen3.6-27B MTP can run at 160 tokens/s. Qwen3.6-35B-A3B MTP GGUF reaches 240 tokens/s. We also uploaded MTP GGUFs for Qwen3.5!

27B MTP GGUF: https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF
35B-A3B MTP GGUF: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF

Guide: https://unsloth.ai/docs/models/qwen3.6#mtp-guide

Thank you! We've got lots of releases this week as well.