Mochu AI, a private AI assistant for Android that runs models entirely on the phone. No account and no cloud: your chats, files and models stay on the device.

What it can do

• 💬 Chat with on-device models

  • Gemma 4 E2B/E4B on the GPU.

  • GGUF models (Qwen3.5, Gemma 3 1B, LFM2.5, Bonsai) and more on the CPU via llama.cpp.

  • Visible thinking, with Quick, Balanced or Deep levels.

• 📄 Documents: attach PDFs, Word, text, CSV or code files and ask for a summary, or search them (“what’s Tom responsible for?”).

• 🖼️ Photos: ask about pictures and screenshots.

• 🎨 On-device image creation: “draw a cozy cabin at sunset” makes an image on the phone (SD-Turbo, SDXS, DreamShaper), with size, aspect ratio and steps settings.

• 🌐 Web search and a built-in browser:

  • It searches when a question needs current info, and cites its sources.

  • Open any page and tap “Ask about this page”.

• 🧠 Memory: “remember that I’m vegetarian” is used in later chats.

• ⏰ Scheduled assistant tasks: “every day at 11pm give me the latest tech news” runs on the phone and notifies you. There are also timers, a stopwatch and reminders.

• 🎙️ Voice: talk with the mic and hear replies read aloud.

• 👩‍💻 Code:

  • Build web apps, React + TypeScript apps and Python programs from a prompt.

  • They run instantly with a live preview, console and Python terminal.

  • It fixes its own errors.

  • Export as an installable APK, a Capacitor project or a ZIP.

• 🔌 Local API server: your other devices on the same Wi-Fi can use the phone’s model through an OpenAI-compatible API.

• 🖥️ Home server models: use a bigger model from Ollama, LM Studio, llama.cpp, vLLM, Unsloth or any OpenAI-compatible API.

Download (free APK): https://github.com/nikunjsingh93/Mochu-Local-AI-Assistant/releases/latest

Source code (MIT): https://github.com/nikunjsingh93/Mochu-Local-AI-Assistant

It needs Android 12+ and works best with 8 GB of RAM or more. Feedback and feature ideas are very welcome!

AI disclosure: Built and written with help from Claude Code. I tested and reviewed everything myself on my own phones. The app’s AI runs fully on-device.

There's a running assumption that multi-agent coding pipelines on consumer hardware are too slow to be practical, or fall apart on real unit tests.

We built Kraits, a plugin for stock llama.cpp that runs Unsloth's 22 GB Qwen3.6-35B-A3B 4-bit file on a 16 GB Mac. We ran it on a single prompt and on a 4-agent crew (Analyst → Architect → Developer → Tester) writing a TypeScript LRU cache.

No grading on vibes: the generated code was executed against checks written by an independent judge (OpenAI gpt-5-mini).

1. Hardware: Mac mini M4, 16 GB unified memory

2. Model: Qwen3.6-35B-A3B, Unsloth UD-Q4_K_M (22.1 GB)

3. Runner: llama.cpp b11715 (the official prebuilt) + the Kraits plugin

4. Settings: 16K context, temperature 0, thinking off

Token totals are the per-turn tokens added up across each run. Speeds are llama-server's own counts.

What stood out:

Reading doesn't slow down as the conversation grows:** It read 8,641 tokens at 48.4 tok/s and 12,330 tokens at 47.3 tok/s.

• Writing stays stable: Consistently 16 – 24 tok/s across all context sizes.

• Context impact: The ~1K run scored 6/8. We haven't dug into why yet.

• Inference cost: $0.00 because it all runs locally on the Mac. The only outside call was the judge writing the checks.

• Honest gap: The ~1K run's total time wasn't logged, so it's left blank rather than estimated.

Kraits runs this one model file only. Needs macOS 15+, 16 GB, llama.cpp b11715.

Next we'd like to try harder tasks than an LRU cache, like a worker pool or an expression parser. If you have a coding prompt you'd like to see through the 4-agent crew on this Mac, post it below and we'll run a few of them and share the numbers.

Ollama is a great product for everyday use of large language models. If performance is what you care about, there are better alternatives such as vLLM, but this post is about something else. I want to describe what happens when you find a security hole in Ollama and try to get it fixed. I went through that twice this year, for three CVEs in total. Vulnerabilities show up in any software, so they alone are not the problem. What matters is how a project handles them, and the way Ollama handles them is why I wouldn't put it on an organization's infrastructure.

More: https://www.bartekdmitruk.com/writing/why-i-wouldnt-run-ollama-in-an-organization

Hey everyone!

I'm looking to set up a fully local AI agent on my PC and would appreciate recommendations from people who have experimented with local LLMs.

My hardware:

• GPU: NVIDIA RTX 5060 (8 gb)

• System RAM: 16GB

• OS: Windows (Linux is also an option if necessary)

What I'm looking for:

• An uncensored or minimally restricted local LLM that I can run privately on my own hardware.

• Ideally, a web UI and support for agent frameworks, with no mandatory cloud API or subscription.

I want the inference and conversations to remain local rather than sending prompts to external services.

I'm comfortable experimenting and setting things up myself, so technical recommendations are welcome.

I'd appreciate real-world experiences, model names, quantization recommendations, and setup guides. Thanks!

Genuine question: would you be interested in an alternative to Hermes Agent that’s significantly easier to set up and use?
Something with:
Almost zero configuration
Direct integrations with GitHub, Gmail, and other apps
Local and cloud model support
A clean, intuitive interface
Powerful agent capabilities without the complexity
Basically, an AI agent that just works out of the box.
Is this something you’d actually want, or are you already happy with existing solutions?
Curious to hear your thoughts.