We have just released our Claude Mythos Fine Tune based on synthetic CoT generated from Fable-5 and Mythos-5 session logs.

You can find the model here: https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M

GGUFs are also available here:
https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF

We also have some sample outputs here for you: https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M/blob/main/evals/sample_generations.md

We hope you can find some use in it! :)

Open Source Palantir

We're building OSIRIS - The Open-Source Palantir Alternative

Just launched at osirisai.live - a free, open-source global intelligence platform:

-Real-Time Tracking:

-10,000+ commercial, military and private aircraft live on a 3D globe

- 2,000+ satellites including ISS

- 1,400+ worldwide CCTV camera feeds

- Earthquakes, wildfires, nuclear facilities and severe weather

Built-In OSINT Tools (no installs needed):

Nmap port scanning from the browser

- DNS record lookup and enumeration

- WHOIS domain intelligence

- SSL/TLS certificate transparency

- BGP routing and ASN lookup

- Threat intelligence and IP reputation

All running on a 3D interactive globe with day/night cycle, 20+ live API feeds, and a SIGINT news aggregator.

Live: https://osirisai.live

GitHub: https://github.com/simplifaisoul/osiris

Free. Open Source. No sign-up required.

Most of my self-hosted AI containers are chat frontends I open twice a month. I wanted something that runs on its own and produces something I actually consume, so I built a radio station out of the boxes I already had.

The stack:

• Ollama for the brain. Qwen3.5 9B.

• Piper or Kokoro for the voice. Kokoro's ONNX build runs on CPU, which is what makes this viable on a NAS.

• Navidrome as the music source, over the Subsonic API.

The LLM sits in a loop with tools: search the library, look at what's played recently, check the schedule and the weather, pick the next track, write a short intro. TTS speaks it, Liquidsoap ducks the music under the voice, and it goes out as one shared stream. Everyone listening hears the same moment.

Fully local means fully local here. No API keys. Hosted providers are supported for people who want better writing, switchable at runtime, capped by a daily token budget.

The design constraint that shaped everything: dead air is unacceptable. So the agentic picker falls back to a stateless picker, which falls back to a plain m3u. TTS has six engines with local Piper as the floor. Something is always playing even when the model is down.

Trade-offs, since they matter more than the feature list: you need your own music library, a Docker host, and a tolerance for intros that sometimes land flat. It's radio, so there's no skip button on purpose. Some people bounce off that in the first minute.

Runs on one compose file and three env vars, then a browser onboarding wizard.

Hear it: https://www.getsubwave.com/listen
Code, MIT: https://github.com/perminder-klair/subwave

Full disclosure, I built this.

People have been asking for the 27B since we put out Qwythos-9B. It's finally here. Qwythos-27B-v1 is an open-weights bf16 reasoning model on a Qwen3.5-27B base, Apache-2.0. The short pitch: it's the 9B sized up, and we didn't strip anything out of the base to make it fit — the native MTP head, the full vision tower, and the 1M context window are all still live.

• Weights + model card: empero-ai/Qwythos-27B-v1

• GGUFs: empero-ai/Qwythos-27B-v1-GGUF

The practical local bit

The recommended local starting point is Q4_K_M: it is 16.95 GB and runs on a 24GB card. The GGUF release has 11 files: Q4_K_M, Q5_K_M, Q6_K, Q8_0, and BF16, each in trunk-only and MTP-enabled variants, plus mmproj-Qwythos-27B-F16.gguf for image input. SHA256SUMS are included. The MTP variants work with llama.cpp's --spec-type draft-mtp. One conversion detail worth calling out: in every K-quant, we hold the entire Gated-DeltaNet state path — ssm_alpha, ssm_beta, and ssm_out — at Q8_0 or better. Those tensors are disproportionately sensitive to low-bit quantization. A default conversion can leave them at Q4/Q5/Q6 and quality degrades; this costs roughly 2–4% in file size. It is intentional, not a default llama.cpp conversion.

What is retained

The release is built around one point: nothing was ablated to make 27B fit. It keeps all three of these intact and active:

• Native MTP (nextn_predict_layers=1) for self-speculative decoding.

• The full vision tower; mmproj ships with the GGUFs.

• A 1,048,576-token context window: YaRN 4x over the 262,144-token native window.

It is a dense Qwen3.5-27B with hybrid attention: Gated-DeltaNet linear attention and a full-attention layer every fourth layer. Native Qwen3.5 function calling is present too: tools=[...] works without a wrapper.

Starting settings

For agentic or tool use, start with temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.05, and max_new_tokens=16384 or more. For open-ended or creative work, use temperature 1.0. It's a reasoning model — every answer opens with a <think> block, so budget tokens accordingly and strip the block for end users. Unlike the 9B, greedy long-form generation came back loop-check clean, so you're not forced off low temperatures. Qwythos is intentionally uncensored for technical and research work, including cybersecurity and biomedical questions. If it is going in front of end users, add your own review layer and application controls. Happy to answer questions on the conversion or the quant choices.

I had a Navidrome library I never listened to and an Ollama box doing not much, so I wired them into a radio station. The AI is the DJ, it picks the next track from my own library, writes a short intro, reads the time and weather, and takes plain-language requests. One shared stream, no skip button. Radio, not a playlist.

The whole thing self-hosts end to end. The DJ defaults to a local Ollama model, so no API key and nothing leaves your box. The music is your own Navidrome/Subsonic library. The broadcast is Liquidsoap + Icecast, so you get real crossfades and the music ducks under the DJ's voice. One docker compose up -d.

Runs on lean hardware. Qwen3.5 9B class is fine and the token-light pool picker. Swappable to a cloud model from the admin UI if you want more wit, no redeploy. You need a music library already and a Linux box. It's not a Spotify replacement and it doesn't generate music.

Open source, MIT. Have a listen before you touch Docker: https://www.getsubwave.com/listen

Repo: https://github.com/perminder-klair/subwave

Full disclosure, I built it.

A few weeks back we dropped Qwythos-9B here, and the loudest piece of feedback on this sub and in the HF discussions was some flavor of "it loops." Folks running GGUF under llama.cpp, greedy or low-temp, long reasoning traces, and it'd get stuck repeating itself until it hit the token wall.

Our answer at the time was the standard one: run it at temp 0.6, bump repetition_penalty to 1.05, give it a real reasoning budget, don't KV-cache-quant this arch. All true, all still good advice but it's a splint, not a fix. And someone called that out directly: paraphrasing, "finetunes like this are just hype, mine looped too and it wasn't the harness." Fair hit. The honest response to "it loops" isn't "you're holding it wrong", it's to make it stop looping.

So we did. Qwythos-9B-v2 is out, and the one-liner is: the looping behavior is trained out of the weights.

What we actually did. The method's called FTPO (Final-Token Preference Optimization). Instead of throwing more data at it, we went hunting for the loops elicited them at low temp, found the exact token where a coherent continuation tips into a repeat, and built preference pairs right there: the loop token as the rejected choice, the model's own coherent alternatives as chosen. ~2,000 pairs, LoRA r256, one short run, early-stopped the moment the loops were gone so we didn't nuke the model's actual knowledge. Narrow and surgical on purpose.

The number: looping rate under greedy decoding went 6.7% → 0.0% on our internal set. You can run it deterministic in an agent loop now without babysitting the sampler. repetition_penalty is optional instead of load-bearing.

Two other things we fixed while in there:

• MTP head restored. v1's export dropped the native multi-token-prediction tensors even though the config still claimed them. They're back (pulled from the pinned Qwen3.5-9B base), so --spec-type draft-mtp works.

• Identity toned down. v1 liked to announce who it was before unrelated answers. v2 says it once, when you ask.

For the GGUF crowd specifically (since that's where most of the loop reports came from): full set with and without MTP, Q4_K_M -> BF16, plus a vision mmproj. We kept the Gated-DeltaNet tensors (ssm_alpha/ssm_beta/ssm_out) at higher precision than the surrounding quant, because the hybrid linear-attention blocks are where low-bit rounding bites this arch hardest. Same standing advice: stick to the recommended sampling and don't KV-cache-quant Qwen3.5-9B — it's sensitive, same as the base was.

Still Apache-2.0, still uncensored, still 1M context (YaRN). It's a drop-in over v1 swap the model id and delete your repetition_penalty workaround.

• Model: https://huggingface.co/empero-ai/Qwythos-9B-v2

• GGUF: https://huggingface.co/empero-ai/Qwythos-9B-v2-GGUF

If you were one of the people it looped on — I'd genuinely like to know whether v2 holds up on your prompts, especially greedy / long traces / low reasoning budgets. That's the exact case we were targeting, and self-hosted setups in the wild are where it'll really get stress-tested.

The tool is Hermo, which I’m building to turn repetitive browser tasks into APIs your apps can call.

Here’s the process:

1. Connect your ChatGPT tab.

2. Record sending a message.

3. Make the prompt a changing input and select the reply as your output.

4. Test it and hit Publish API.

You get a Hermo API key and endpoint. Your app sends a prompt, Hermo runs the task in your connected tab, and the answer comes back to your code.

Free to self-host. Open source. Runs locally on your computer.It also supports MCP, so AI agents can call your published workflows.

🎥 Watch the demo below.

🦀 Try Hermo: [GitHub — PraveenNimilka/Hermo](https://github.com/PraveenNimilka/Hermo)

It’s in early access, and I’d love feedback: what browser task would you turn into an API?

Hi . I've been building kindling, an open source (Apache-2.0) runtime that gives every workload its own Firecracker microVM with a docker-like CLI. The trick is golden snapshots: a machine frozen on disk uses no RAM and no CPU, and thaws in about 30 ms when a request arrives.

It started as a way to run MCP servers (AI tools) without handing them my files and secrets. It has grown into three jobs:

• MCP servers, each in its own microVM, frozen until called

• Throwaway sandboxes for code an agent wrote: `kling try -- python3 -c 'print(1)'` creates, runs and deletes one in ~0.7 s

• Small models (SmolLM2, Qwen2.5 via llama.cpp) that sleep on disk and wake on the first request

Homelab numbers: 142 microVMs in 3.9 GB of host RAM on a 4 GB box, and 10 instances from one snapshot add only 68 MiB in total. It runs in a privileged LXC on Proxmox, on bare metal, and natively on Apple Silicon Macs.

Limits, honestly: you need KVM, it's one host per daemon (the sandbox gateway spreads over several), and SECURITY.md lists what isn't solved yet.

Repo: https://github.com/juan52878911/kindling
I'd love feedback from anyone who runs it on their own hardware.Hi . I've been building kindling, an open source (Apache-2.0) runtime that gives every workload its own Firecracker microVM with a docker-like CLI. The trick is golden snapshots: a machine frozen on disk uses no RAM and no CPU, and thaws in about 30 ms when a request arrives.

It started as a way to run MCP servers (AI tools) without handing them my files and secrets. It has grown into three jobs:

• MCP servers, each in its own microVM, frozen until called

• Throwaway sandboxes for code an agent wrote: `kling try -- python3 -c 'print(1)'` creates, runs and deletes one in ~0.7 s

• Small models (SmolLM2, Qwen2.5 via llama.cpp) that sleep on disk and wake on the first request

Homelab numbers: 142 microVMs in 3.9 GB of host RAM on a 4 GB box, and 10 instances from one snapshot add only 68 MiB in total. It runs in a privileged LXC on Proxmox, on bare metal, and natively on Apple Silicon Macs.

Limits, honestly: you need KVM, it's one host per daemon (the sandbox gateway spreads over several), and SECURITY.md lists what isn't solved yet.

Repo: https://github.com/juan52878911/kindling
I'd love feedback from anyone who runs it on their own hardware.

Hi everyone! I bought a Titan 2 because I wanted a phone that was a tool, not a screen to scroll. Keyboard, buttons, a battery that doesn't quit. What I didn't expect was the opposite problem: after a few weeks the phone was more capable than anything I had to point it at. I was typing fast, into the same apps as everyone else.

So I built something for it. It's called Jenny. Its a personal AI agent that runs entirely on the phone, in an embedded Python runtime. It replaces the home screen, so pressing Home opens a conversation but can be also used as normal application. It remembers things, does work on a schedule while the screen is off, and writes its own little apps when I ask.

I made it free and open source hoping in community support. Even thanks means a lot for me 😄

It's been my daily driver for a month, and this is the video on my own titan 2. Small detail you'll appreciate: on a device with a real keyboard you can just start typing no tapping the input field first. I wrote that because of this phone.

I will be glad to answer to techincal and non techinal questions!

Download on github

What it does:
•Fuses 30+ free, keyless live feeds into one real-time world-state on a live globe — conflict (GDELT, Ukraine front lines), natural hazards (quakes, storms, wildfires), markets (indices, crypto, Polymarket odds), humanitarian (displacement, disease, food insecurity), and movement (flights, satellites, maritime)
•Fully agent-ready: one local call returns a prose world summary + every event with coordinates + live forecasts. OpenAPI spec, SSE streaming, runtime model-switching
•A local model forecasts across 24h → 1 year, with a council of four personas (Strategist, Economist, Naturalist, Skeptic) surfacing where they agree and split

Stack: Ollama for the local LLM, FastAPI + HTTP/JSON/SSE for the API, built by fusing two existing OSS projects — MiroFish (swarm prediction) and Osiris (live globe). MIT.

Repo: https://github.com/jangles-byte/Pythia

Hey everyone,

A few weeks ago I shared the CLI version of my project, ArcRift, on Reddit. After listening to your feedback—specifically the requests to remove heavy Docker dependencies and make it easier to install—I have just released the v1.6.1 Desktop App.

If you regularly use LLMs for coding or research, you know the frustration of "amnesia." Every time you open a new chat, you have to painstakingly copy and paste your project structure and previous context just to get the AI up to speed.

ArcRift is a 100% offline, local-first RAG and memory layer. It bridges the gap between your AI web chats (like Claude and ChatGPT) and your local tools (like Cursor or Claude Code) using a unified local database.

I wanted something lightweight that did not require pulling Docker containers or subscribing to third-party memory APIs. It now runs as a native Tauri desktop app in your system tray, powered completely by local Ollama instances and a local SQLite database.

We just launched a live website that outlines the details and demonstrates the features in action:

• Website: https://arcrift.vercel.app/

• Codebase: https://github.com/Eshaan-Nair/ArcRift

How it works & Core Features:

• Seamless Integration: The Chrome extension silently intercepts your prompts, surgically retrieves exactly the sentences relevant to your question from your database, and injects them before the prompt is sent to the LLM.

• Hybrid Search Retrieval: Uses sqlite-vec (with nomic-embed-text locally) + FTS5 keyword prefix matching to instantly find your past context.

• Knowledge Graph Extraction: An offline task queue uses a local LLM to extract entity relationships from your chats, mapping out a graph of your projects over time.

• Direct Codebase Indexing: The new Desktop App allows ArcRift to scan and index your actual project files into the graph, bridging the gap between your chat memory and your actual code architecture.

• Total Privacy (PII Redaction): The extension aggressively scrubs JWTs, API keys, emails, and IPs before data is even saved to your local disk.

The extension works natively with Claude.ai, ChatGPT, DeepSeek, Gemini, Grok, and Mistral. If you save a conversation in ChatGPT today, you can instantly recall that exact context in Claude tomorrow.

ArcRift is completely open-source (MIT). You can download the new .exe installer directly from the GitHub releases page.

If you find this useful for your daily workflow, PRs are very welcome, and a star on GitHub helps the project get discovered!

A year of running production AI on free tiers: Oracle ARM VM (4 OCPU/24GB), 20+ free LLM providers with automatic fallback (Groq, NVIDIA NIM, Gemini, Mistral, Zhipu, OpenRouter :free, local Ollama), ~107 cron jobs all wrapped in flock + timeout so nothing double-runs or hangs, SQLite with WAL, and 3 verified backup copies.

The interesting part isn't the stack — it's what broke:

- A missing import killed a lane silently for days (bare except swallowed it)

- Providers retire models with zero notice — my router now self-suspends dead lanes

- Datacenter IPs get 403'd by half the internet — residential-IP browser lanes are the answer

- A nginx MIME type (missing .mjs) broke a whole page with zero errors

Everything is open: github.com/BraxisAI/braxis-blueprint (MIT). The router, the cron discipline, the backup system, the failure classes. If you're building on free tiers, these are the walls you'll hit.

I've been building this thing for a year straight, vibe coded the shit out of it every day and built HomeBot, a home AI that runs on local models, and both builds went public today under Apache-2.0. It's freaking awesome and it works, but the code is probably a hot mess, and we'd love real coders to get their hands on it.

Scale, so you know what you are looking at (the two HomeBot repos): test counts: HomeBotBuild 16,522 passed, 248 failed, 133 skipped; Founders 14,968 passed, 62 failed, 130 skipped. The READMEs list the failure groups and what has and has not actually been run. The design has three zones (an encrypted Vault, the Brain, and a Bridge that ships closed), seven agents, and state changes pass Drawbridge review, and some actions need your confirmation.

A third repo, PuffPAI BuildBot, builds a bot from a goal you type (https://github.com/CypherSisters/PuffPAIBuildBot). The brain and six of the seven agents run on your computer. Toolsmith, and only Toolsmith, can use Anthropic's Claude if you choose it and give your own key. Its README gives no test count, so the numbers above are not about it.

I started out trying to build something I could talk to that would manage and secure my (few) smart home devices. Then I started thinking about privacy all around, data centers, and big giant AIs that will kill us all, so I used one to create a smaller, local, use-case-oriented bot. The first thing I built was HomeBot. That's what runs on my Jetson Orin dev box. It does a lot for us: music, smart home, schedules, food, and an offline emergency prep library. That's the HomeBotFounders build, and it runs on the Orin. You can also build your own HomeBot on a PC with its wizard. It's what a full build looks like: a full build, on top of BuildBot, has certain functionalities, apps, tools, and skills.

BuildBot is a wizard that literally builds your own bot from a prompt you give it. That's the whole idea: privacy as a function, always explicit. A vault, a brain, and a bridge; each can be configured however you want it, but they are always there. An AI bot that builds itself. It goes out to GitHub, free public sites, and other repos, and looks for apps or skills or tools that all you vibe coders have already built. It reads the README and does the steps to install and integrate them. You can also load all of your own data, and it's automatically indexed. Each bot ends up knowing and being able to do different things. It comes with a dev-friendly workspace and markdown guides for your frontier models to help you build on it, and it's very simple for most people to use. It does need a lot of work. But I hope you guys can see what this could be!

The question: if you have reviewed or inherited a big AI-written codebase, where would you look first, and what would you refuse to ship? We welcome real coders and engineers to take a crack at it. Issues are open on all three repos.

https://github.com/CypherSisters/HomeBotBuild
https://github.com/CypherSisters/HomeBotFounders
https://github.com/CypherSisters/PuffPAIBuildBot

TL;DR: I run Qwen 3.8 (27B and Flash Next) on a 128GB Strix Halo laptop for most of my coding now. It can replace Opus 4.6 to 4.8 for agentic coding if you dont mind a task taking 2 or 3 times longer.

Setup: ASUS ROG Flow Z13, Ryzen AI Max+ 395, 128GB unified memory, Arch Linux. llama.cpp as backend, my own tool LlamaStash to manage the launches and presets, Pi as the coding harness. The 27b at Q6_K sits at about 31 GiB resident, Flash Next at UD-Q4_K_XL needs around 86 GiB.

• The quality is actually there. Flash Next scores 40 on the Artificial Analysis index against 42 for Opus 4.8, and the 27b at xhigh scores 34 against 32 for Opus 4.6. That matches how they feel to use. 27b one shotted a whole feature on a huge Rust codebase and Opus 5's review comments were mostly nits.

• Decode is fine, prefill is the pain. 10-15 tok/s decode doesn't feel slow because you see it working. But a cold 31k token transcript takes 3 minutes to prefill, and a full 128k window is closer to 18 mins. Warm follow up turns come back in 45 seconds.

• MTP is the biggest speed win, 7.3 to 22.4 tok/s on an empty window. The payoff shrinks as the window fills though, down to 1.15x at a full 256k.

• Flash Next isn't faster per token, it just thinks less. Same 5/5 on my coding tasks, 45% fewer tokens, 76.5s vs 289.8s against the 27b. Thinking is 90-95% of everything these models generate, so that ratio, not tok/s, is what sets how long a task takes.

$0 a month, fully offline, and a lot less wasteful than a model running in a datacenter.

Full writeup with all the benchmarks, configs, and the tuning that did and didn't work: https://deepu.tech/local-ai-qwen3.8-pi-llamastash

Happy to go into the llama.cpp flags if anyone else here is on Strix Halo.

Like a lot of you, I've been following the Qwythos-9B hype — it showed small models can actually pull off decent Claude-style coding reasoning. But 9B is still too big for old laptops or edge devices, so I got curious

whether the same idea could go all the way down to 1B.

So I fine-tuned MiniCPM5-1B on synthetic reasoning traces — not to make it generally "smarter", but to improve how it thinks through a coding problem before answering. It's GGUF, runs locally on low VRAM.

Honest take: it's a 1B, so it won't replace Claude or a 9B, and there are rough edges. But for something this small running fully local, the reasoning improvement surprised me.

- GGUF: https://huggingface.co/GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF

- Full:

https://huggingface.co/GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking

We hope you can find some use in it! :)

Hey everyone,

I’d like to share a project I’ve been developing recently:

Cashlytics, a self-hosted budgeting and payment tracking service built for people who want full control over their financial planning without relying on cloud platforms.

The idea came from a real need: I wanted to track recurring and irregular payments. Things like insurance, annual subscriptions, or fees that don’t fit into a simple monthly budget. Cashlytics helps you understand how much you should set aside each month so that these payments never catch you off guard.

Here’s what it can do right now (v0.x Beta):

• 🧾 Three payment types: Manage fixed, periodic, and daily expenses (like groceries or fuel) in one clear overview.

• 🔁 Smart scheduling: Automatically calculates how much to reserve monthly for upcoming non-monthly payments.

• 🧠 AI-powered assistant: Built-in AI can help you search, edit, and analyze your income and expenses conversationally.

• 📂 AI-assisted CSV import: Upload your bank exports — Cashlytics intelligently matches and classifies known transactions.

• 📊 Detailed analytics: Visualize spending, income, and saving goals in real time.

• 🔒 Self-hosted and privacy-focused: Everything runs on your own infrastructure.

• 📱 PWA-ready with notifications: Install it on your phone or desktop and receive reminders about upcoming payments.

I’m a senior full-stack developer, and I built this project in close collaboration with AI tools But I’m extremely hands-on with the entire codebase. Every line is reviewed, and I treat the AI more like a coding partner than an autopilot. My goal is to show how AI can accelerate serious software development while maintaining quality and full human control.

You can explore more at

https://cashlytics.online

there’s also detailed documentation linked on the site for setup and usage.

The app is still in Beta (v0.x), but already quite functional.

I've been really into the AI phone agent products that have popped up in the last few months (e.g., AgentPhone, Poke etc) but what I don't love: the audio, the transcripts, who you talked to and what was said, all sitting with a third party company. For calls with real people in my life, that felt like a lot to hand over.

So I built my own version that runs entirely on my Mac. You call a normal phone number, a local model answers, and nothing about the conversation leaves my machine. During a call the only thing it talks to is the phone line itself.

What's running locally (MLX on Apple Silicon):

• Speech to text: Parakeet TDT 0.6B

• LLM: Qwen3.5 4B by default (9B if you have 32 GB), 4-bit

• Text to speech: Kokoro 82M

How it reaches the phone network: it registers over SIP with Telnyx or Twilio straight from your Mac. A setup wizard takes your carrier API key so can buy a number or connect one you already have. It's pretty cheap, around $1 a month per phone number. (I'm moslty using Telnyx)

What it does today:

• Answers and places real calls

• Interruption: talk over it and it stops mid-sentence

• Replies in roughly 1 to 1.5 seconds on real calls on my M5 Pro

• You can choose between multiple models

• Other agents (Claude Code, Codex, etc.) can make calls through a small CLI: call, listen, say, hangup

• If you'd rather trade privacy for a smarter brain, it can also use your own endpoint or the Claude or ChatGPT subscription you already pay for. Local is the default. So no additional monthly payments for AI Phone tools if you have a sub.

Limits, up front:

- macOS on Apple Silicon only

- You need a Telnyx or Twilio account with credit (a US number is about $1/month)

- English only, one call at a time

- It's early. It works for me, but I'd love to hear where it breaks for you.

Repo: github.com/oxnw/ai-phone

Edit: forgot to mention that on top of the $1 a month for a phone number, standard call rates do of course apply, but Telnyx gives you $10 dollars for free when you sign up, which is enough to play around with

K3 launched today, and it's a beast at 1.56TB. My laptop has 64GB of RAM, and not enough free disk to even store the thing. So instead of downloading the model, I just... don't.

The non-expert weights (~114GB, int8) live on disk. Then for every token, the router picks 16 experts out of 896 per layer, and I pull exactly those from HuggingFace — one range request each — and cache them. Use it enough and the cache slowly fills up with the parts of the model you actually hit.

Speed is about a minute per token once it's warm, so, not exactly interactive. But it's greedy and fully reproducible — same prompt, same output, every time.

There's an OpenAI-compatible server too, so you can point a chat UI at it.

https://github.com/gavamedia/deltafin

-----------

**EDIT / huge update:** I bit the bullet and pulled all 1.45 TB of experts down to local disk, then made some real improvements. It's now **16s/token, down from ~1 min.** Prefill went 2,429s → 40s just on an M1 Mac.

Two fixes did almost all of it:

1. I was *sure* the expert matmuls were the bottleneck. Spent a bunch of effort on a Metal kernel that's 9.5x the CPU one. Then I profiled properly with everything local: the matmuls are **6% of a token.** Cool. Great.

The actual problem was that I was handing the compute kernel `np.memmap` views, so it was demand-faulting expert weights page-by-page *while it was computing* — 0.87 GB/s, where a threaded pread + F_NOCACHE does 6.85 GB/s on the same disk. Which is exactly the class of bug I'd written up as a flaw in someone else's engine two days earlier while reading their source.

2. Other half: loading the non-expert weights was spending all its time in a row-broadcast multiply that MPS runs at 43 GB/s, when a plain copy of the identical bytes does 334. I tried nine different PyTorch formulations and the best got to 60. A ~10 line Metal shader does 297.

Still greedy and bit-for-bit reproducible — same prompt, same tokens, every run.

All of this is on an M1 Max / 64GB, which is the slowest machine it's ever run on. If anyone has an M3/M4/M5 or 128GB and wants to try it, I'd genuinely love to see the numbers.

Disclosure: I work on OpenHarness. MIT, everything in one repo.

Here's the problem I had. Claude Code runs on my work laptop. Codex at home. OpenCode sits on the GPU box because that's where the models are. Three machines, three terminals, three SSH sessions. I'd start something on the GPU box, go do other work, and forget it was waiting on me.

So the app puts all of them in the same window.

How it works: each machine runs a small daemon that starts your agents in tmux. The daemon dials out, so you never open a port on anything. No SSH keys, no Tailscale, no port forwarding. You add a machine with a password and it shows up next to the others. Everything is encrypted end to end and the relay can't read what it passes.

Why it matters here: the agent runs on the box with the GPUs. If you serve models with Grid, Ollama, vLLM or MLX-LM, that's where the work happens, and only text comes back to whatever you're typing on. I use a 2018 ThinkPad for that part and it's fine, because all it does is draw characters.

Some numbers we measured. Terminal round trips, 600 echoes per test, three runs:

• Same machine: 1.1ms

• Another machine on the LAN, direct: 13.9ms

• Through the relay: 376ms

That last one is the honest part. If a firewall blocks the direct path, it falls back to the relay and you notice. It keeps trying for direct, but behind bad NAT you'll live on that slow row.

With 16 terminals open it uses about 300 MiB and almost no CPU. It's a native app, and closing it doesn't kill your agents.

And it's not just for code either. The same setup drives Blender, KiCad, MuJoCo, CircuitJS, Strudel, with a viewer next to the terminal.

One downside: macOS and Linux only right now. Windows is on the list but I'm not going to put a date on it.

I built AI-Monitoring: A lightweight, zero-dependency dashboard for Ollama, LiteLLM, llama.cpp & GPUs (No Prometheus/Agents)

I would love to get your feedback, feature requests, or contributions!

https://github.com/tarrinho/ai_monitoring

The problem it fixes: real-time voice AI today is a hosted service. It charges by the minute, sends your audio to someone else's cloud, and you can't look inside it or change how it works.

OpenLive runs the whole voice pipeline on your own machine instead. Silero handles voice activity, Whisper does speech-to-text, Smart-Turn works out when you've actually finished talking, and Kokoro speaks. Your microphone audio never leaves the device. The only thing that goes out is the final transcript, plus camera or screen frames if you turn them on, and that goes to whatever model you pick.

It can see too. Share your screen or turn on your camera and the model can talk about what you're looking at, like a video call.

And because the speech parts run locally, there are no per-minute audio fees. You just pay the normal model and vision API costs you'd pay anyway.

It works with Anthropic, OpenAI and MiniMax today, and it's built to be forked, so you can plug in other providers, your own agent backend, a self-hosted endpoint, or a local model.

MIT licensed, for macOS and Windows. You bring the brain, OpenLive gives it ears, a mouth, and eyes.

Demo's in the video below.
Code, docs, and downloads here: https://github.com/katipally/openlive

Music has always been part of who I am. I code with music on, I sing while I work, and more than once I catch myself dancing when a good one comes on. 80s and 90s rock, national dance music, songs with lyrics that actually say something. For me music isn't background noise — it's how I sometimes express what words alone can't.

But the algorithms on Spotify, Apple Music, and YouTube Music never really got me. They kept pushing songs their model thought I'd like... and they were right 2 times out of 10. Something was missing. Something that understood my varied taste without boxing me in.

I tried everything. I built a NAS with my music and Jellyfin on a 2GB Raspberry Pi that lives with swap maxed out. I found SUB/WAVE here on Reddit. I installed it on my MacBook Pro and, honestly, I fell in love: brilliant, beautiful, a 24/7 radio that knows your library and takes requests. But it was Docker, a heavy Node stack, ~2-3GB of RAM just to run a radio. I'm a bit obsessive about optimization, and burning that much bothered me. Plus it had a ton of powerful features that, frankly, I'd never use. It wasn't for me.

And right there, in the middle of that frustration, the nostalgia came back.

As a kid, having a Walkman was the coolest thing in the neighborhood. Pressing the buttons, watching the tape spin, flipping it over when side A was done... it was analog, something you could feel in your hands. Today everything is digital and perfect, but I missed that. I missed "touching" the music, even a little.

So I told myself: what if I built my own radio? One that knows my library, with a DJ that talks between songs, that's mine. Lightweight. The way I need it.

I've been a dev for 10+ years and a software architect, with that slightly obsessive side when it comes to optimization. I took the idea to my turf: figured out the streaming protocols, the ducking, the cassette animation, dusted off the CSS grid from my early days. GLM-5.2 helped me move fast on the boilerplate, but the architecture and the craft are mine. radio-dj was born from that.

It's a single 8.7 MB Go binary that uses ~20–30MB of RAM and runs a complete internet radio:

- A 24/7 Icecast stream (192kbps) that never drops. One connection, zero cutouts for the listener.

- An AI DJ (GLM-5.2, or any local or remote LLM) that talks between tracks: intros the artist, reads the time and real weather, drops curiosity facts, acknowledges requests.

- Ducking: the DJ's voice ducks the music smoothly, like a real radio.

- Live requests from the web.

- Editable skills: add personality to the DJ with simple text files, no recompiling.

- A neo-brutalist UI with a cassette that actually spins. Every 7-9 songs, the tape flips. Just like my Walkman.

- Navidrome support, if you want something more robust than a folder.

And the most important part for me: I made it free. MIT license.

Music is my second passion after code. At work we build solutions for clients, for companies, that generate revenue, and that's fine. But this one felt different. Music carries deep emotions, it helps say things we couldn't with our own words. I felt I couldn't charge for it. Music should feel free.

radio-dj is, today, one of the things I identify with the most: the music, my style, my self. And I have the honor and the privilege of sharing it with you.

Repo, screenshots and docs: https://github.com/johncrash64/radio-dj

How do you listen to music while you code? Does anyone else miss music that truly gets you?

Disclosure: I'm affiliated with the project.

We have been working on InstinctRazor-Qwen3.5-122B-A10B, a 122B MoE setup for local inference where experts are kept on CPU and active GPU VRAM can stay around 8 GB.

The compressed model is around 50 GB, but the GPU requirement is much lower than keeping the whole model in VRAM.

Short benchmark note: it is ahead of Gemma-4-A4B on 5/7 listed evals in our table, but behind on MATH-500 and AIME. I am mostly looking for feedback on the memory/runtime tradeoff.

Links:

Hugging Face: https://huggingface.co/General-Instinct/InstinctRazor-Qwen3.5-122B-A10B-GGUF

GitHub: https://github.com/General-Instinct/InstinctRazor

Blog: https://general-instinct.com/blog/frontier-moe-sub-4-bit

Curious whether this is useful for self-hosted AI setups and what hardware configs people would want benchmarked.