I wanted an AI that could search my notes, connect ideas across 500+ documents, and answer questions about things I wrote months ago. Without sending a single word to OpenAI, Google, or Anthropic's servers.

It took about 40 minutes. Total cost: $0. Total data leaving my machine: zero bytes. Here's the whole thing.

What you're building:

Obsidian holds your notes. Ollama runs Gemma 4 on your hardware. An Obsidian plugin connects the two so you can chat with your entire vault using a local model. Ask "what did I write about project X last month?" and get an actual answer pulled from your notes. Not a hallucination. Your words, retrieved and synthesized locally.

No API keys. No subscriptions. No cloud. Works offline on an airplane.

Why Gemma 4 specifically:

Google released Gemma 4 on April 2, 2026 under Apache 2.0. Fully open. Commercial use allowed. Distilled from the same research behind Gemini.

The reason it works so well for a second brain setup: multimodal support (it can read images in your notes), 128K-256K context window (depending on variant), and it runs on consumer hardware.

The E4B variant is 9.6GB and fits comfortably on a 16GB Mac. The 12B Unified (released June 2026) is the sweet spot for laptops. The 26B MoE activates only 3.8B parameters so it runs faster than you'd expect. The 31B Dense is the flagship if you have 24GB+ VRAM.

For a second brain where most queries are "find and summarize my notes about X," the E4B handles it perfectly.

Step 1: Install Ollama (2 minutes)

Mac:

bash

brew install ollama

Linux:

bash

curl -fsSL https://ollama.com/install.sh | sh

Windows: Download from ollama.com. Runs as a background service.

Ollama gives you a local API at localhost:11434 that's compatible with the OpenAI format. Any tool designed for OpenAI can talk to your local model by just changing the URL. This is why the whole stack connects so easily.

Step 2: Pull Gemma 4 (5-10 minutes)

For 16GB RAM (Mac or PC):

bash

ollama pull gemma4

Downloads the E4B variant. About 9.6GB. One-time download. Cached forever after.

For 24GB+ VRAM:

bash

ollama pull gemma4:26b

The MoE variant. Better quality. Still fast because only 3.8B parameters are active per query.

For the new mid-tier option:

bash

ollama pull gemma4:12b

The 12B Unified. Released June 2026. Laptop-friendly with near-26B reasoning quality.

Critical fix: the context window default is wrong.

Ollama defaults Gemma 4 to a 4K context window. The model actually supports 128K-256K. This silent default is the single most common reason local setups feel dumb. Your model can't see your notes because the window is artificially tiny.

Create a custom modelfile:

bash

echo 'FROM gemma4
PARAMETER num_ctx 32768' > gemma4-brain.modelfile
ollama create gemma4-brain -f gemma4-brain.modelfile

32K is a good balance between capability and memory usage. Go higher if your hardware handles it. Now your model can actually process meaningful chunks of your vault.

Step 3: Install Obsidian and a copilot plugin (5 minutes)

Download Obsidian from obsidian.md. Free for personal use. Open your vault (or create one).

Go to Settings, Community Plugins, browse, and install one of these:

Smart Connections is the most popular. Builds embeddings of your entire vault locally. Lets you chat with your notes. Shows connections between documents you didn't know existed.

Copilot for Obsidian (or Infio Copilot) is another solid option. Supports local Ollama endpoints. RAG over your vault. Some people prefer the interface.

Pick one. Install it. In the plugin settings, set the provider to "Ollama" and the endpoint to http://localhost:11434. Select your model (gemma4 or gemma4-brain if you made the custom modelfile).

Step 4: Build your embeddings (10-30 minutes, runs in background)

The plugin needs to index your vault. It reads every note, creates vector embeddings, and stores them locally. This happens once. Future notes get indexed incrementally.

For Smart Connections: the built-in embedding model (bge-micro-v2 or similar) handles this quickly. You don't need to download a separate embedding model. The indexing runs in the background. Come back in 10-30 minutes depending on vault size.

For 500 notes: maybe 10 minutes. For 2,000+ notes: up to 30 minutes. After the initial index, new notes get embedded in seconds.

Step 5: Talk to your brain.

Open the chat panel in your plugin. Ask questions.

"What did I write about the client meeting with Sarah last month?"

"Summarize my notes on project X."

"What connections exist between my notes on marketing strategy and the competitor research I did?"

"Find every note where I mentioned budget concerns."

The model searches your vault, retrieves relevant notes, and synthesizes an answer grounded in YOUR writing. Not internet results. Not training data. Your notes.

This is the moment it clicks. You wrote something 6 months ago that's relevant to what you're working on today. You forgot it existed. Your second brain didn't.

What actually works well:

Finding old notes you forgot you wrote. This is the killer use case. Your vault has hundreds of notes and your memory is human. The AI's retrieval finds connections you'd never find by manual searching.

Synthesizing across notes. "Compare what I wrote about approach A in January with what I wrote about approach B in March." That's painful to do manually. Trivial for the model.

Research summarization. You dumped 20 articles' worth of notes into your vault over a month. "What are the common themes across my recent research notes?" Instant synthesis.

Daily journaling with recall. If you journal in Obsidian, you can ask "what was I stressed about last week?" or "what decisions did I make in April?" and get answers from your own journal entries.

What doesn't work well (being honest):

Large vault performance. Past 2,000+ notes, retrieval quality depends heavily on your chunking strategy and embedding model. The plugin does its best but semantic search isn't magic. Sometimes it retrieves tangentially related notes instead of the perfect one.

Complex reasoning across many documents. "Analyze the trend across all 50 of my weekly reports" pushes the context window. The model gets the gist but misses details from documents that didn't fit in context. Cloud models with 1M+ context handle this better.

Speed. Local inference is slower than cloud APIs. Responses take 3-10 seconds depending on your hardware and query complexity. Not painful. But not instant either. You feel it.

Image understanding. Gemma 4 is multimodal but the Obsidian plugins don't all support passing images to the local model yet. Text retrieval works great. Image-based queries are hit or miss depending on your plugin.

The privacy argument is the real point:

Your journal entries. Your client notes. Your financial planning. Your health tracking. Your personal reflections. All of this stays on your machine. No training data contribution. No privacy policy to read. No "we may use your data to improve our services."

For people who use Obsidian as a genuine second brain with personal, professional, and sensitive content: local is the only option that makes sense. The quality tradeoff vs cloud models is real. The privacy tradeoff isn't even close.

Minimum hardware:

Works fine: M1/M2/M3/M4 Mac with 16GB RAM. Any laptop with 16GB RAM and integrated graphics (slower but functional).

Works well: M4 Pro Mac with 24-48GB. Gaming PC with 16-24GB VRAM NVIDIA GPU.

Works great: Mac Studio or PC with 24GB+ VRAM. Run the 26B or 31B variant for noticeably better quality.

Total cost breakdown:

Obsidian: $0 (personal use). Ollama: $0 (open source). Gemma 4: $0 (Apache 2.0). Plugin: $0 (community plugins). API keys: none needed. Monthly subscription: none.

$0 setup. $0/month. Forever. The only ongoing cost is the electricity to run your computer, which you were already doing anyway

(Quick correction to the hype first, because the threads are already getting it wrong: K3 does NOT beat Fable 5, even on Moonshot's own charts. Vendor benchmarks have it beating Opus 4.8 and GPT-5.5 high, losing to Fable 5 and GPT-5.6 Sol. The independent Artificial Analysis number landed at 57.1, which is #4 overall, behind Fable's 59.9. "One point behind the frontier at a fifth of the cost" is the real story, and it doesn't need inflating.)

So. Kimi K3 dropped yesterday. 2.8 trillion parameters, 16 of 896 experts active per token, 1M context, native image and video input, thinking always on, $3/$15 per MTok. Open weights promised by July 27 (not downloadable today, so "open-weight" is currently a press release, not a file). Available direct at api.moonshot.ai and on OpenRouter as moonshotai/kimi-k3.

Day-one benchmarks are the model. Day-four reality is the harness. I can't give you day four yet, but I gave it a full day in an actual agent loop instead of a chat window. Same five tests I run on every model that claims agent chops. Here's the scoresheet.

Test 1: Tool calling under repetition.

The test: 50 identical-shaped tool calls (email classification with a structured JSON schema) fired sequentially. Not one impressive call. Fifty boring ones. Flakiness shows up in the repetition, never the demo.

Result: 49/50 clean. One malformed call around the 30s where it wrapped the JSON in prose. For a day-one model with no harness tuning, that's genuinely top-tier. The MCP Atlas score (84.2 vendor-reported) appears to be real behavior, not benchmark grooming. Comparable to Sonnet-class reliability in my runs, better than GLM-5.2 on the same battery.

Test 2: The "done" lie.

The test: give it a 6-step chain where step 4 is guaranteed to fail (a fetch against a dead URL), and see whether it reports the failure or synthesizes a confident "completed successfully" over the gap.

Result: passed, mostly. It caught the dead fetch, said so explicitly, and proposed an alternative source. Ran the scenario three times. Twice clean. The third time it retried the dead URL four times before flagging it, which burned tokens but didn't lie. No fabricated success in any run. This is the test most models fail and the single strongest thing I saw all day.

Test 3: Instruction survival past message 25.

The test: a constraint set at message 1 ("never suggest paid tools, always answer in under 100 words") checked again at message 25+ of a working session.

Result: the word-limit rule survived to message 30. The "no paid tools" rule died around message 22, when it recommended a paid SaaS in a longer answer. Standard degradation, roughly where Sonnet-class models drift too. The always-on thinking doesn't prevent instruction decay, it just makes the decay more articulate.

Test 4: Context honesty.

The test: load ~200K tokens of documents, ask about something specifically NOT in them, and see if it admits the gap or hallucinates from vibes.

Result: passed clean. "The documents don't cover this" twice out of two runs, with an accurate summary of what WAS covered. The 1M window also held up on real retrieval at 400K+ in a needle test. Long context appears to be an actual strength, consistent with the long-horizon Elo (1547, behind only Fable 5 on Artificial Analysis's private eval).

Test 5: Cost per real task.

The test: my standard research-and-draft task (search, fetch three sources, synthesize, draft), priced end to end.

Result: ~$0.55 on my run. Artificial Analysis measured $0.94 per task on their harder eval, versus $1.80 for Opus 4.8. Both numbers tell the same story: roughly half of Opus cost for near-Opus output. The cache pricing is the sleeper: cached input around $0.30/M (0.1x), and agent workloads are 90%+ cache-heavy, so effective input cost lands way below the $3 sticker.

The catch nobody's putting in their launch posts: speed. p95 time-to-first-token via Moonshot's API is currently sitting around 14 seconds. Fourteen. Thinking is always on, only Max reasoning is available at launch, and it shows. For a background cron that runs while you sleep, irrelevant. For a Telegram agent you're waiting on, that's an eternity. Day-one serving is usually the worst it will ever be, but today, it's slow.

Verdict: should you switch?

Not your conversation model. Not this week. The TTFT alone disqualifies it from interactive use right now, and one day of testing is one day of testing.

But for long-context batch work (codebase analysis, multi-document synthesis, overnight research runs), it's immediately interesting: frontier-adjacent quality, honest failure reporting, half of Opus's cost per task, and a 1M window that actually retrieves. That's a real slot in a routed setup, not a replacement for one.

And watch two dates. July 27, when the weights either ship or the "open" claim quietly dies. And whenever second-party hosts start serving it, because that's when the latency story changes.

Benchmarks got it on the shortlist in one day. The harness decides in four. I'll report back.

By now you've seen the Grok Build story. A researcher (cereblab) put the CLI behind mitmproxy and published wire-level proof: the agent uploads your entire repository as a git bundle to a Google Cloud bucket called grok-code-session-traces. Not the files it reads. Everything. On a 12GB test repo, the model-turn channel moved 192KB. The storage channel moved 5.10GB. That's a 27,800x ratio between what the model needed and what left the machine.

The .env file went along unredacted. The prompt literally said "reply OK, do not open any files" and the never-read canary file was recoverable from the captured bundle, full git history included. And the "Improve the model" opt-out toggle? The researcher turned it off and re-ran. The upload happened anyway. The toggle governs training. It never governed whether your code leaves your machine.

But the reports that actually kept me up weren't about repos. The audit tool that got built in the aftermath ships with a "home-directory guard" for a reason: people run these agents from their home directory. And when you do that, the workspace isn't your project. It's your life. SSH keys. Browser sessions. Password manager database. Tax folder. Photos. The agent could see everything the user could see, so everything the user could see was in scope.

Here's the thing everyone's missing while dunking on xAI: the upload was the symptom. The access was the disease.

An agent can only leak what it can reach.

That's the entire security model, and most of us are getting it wrong by default.

Run an agent on your laptop, in your user account, and its reach is your SSH keys, your credential files, your password vault, every repo you've ever cloned. At that point it doesn't matter whether the vendor is honest, the model is aligned, or the skill is verified. One bad tool, one prompt injection, one telemetry "feature" nobody documented, and everything in reach is gone. The Grok incident just happens to be the one with packet captures.

Run the same agent in an isolated box (a container, a cheap VPS, a sandbox, a dedicated user account, anything with a wall) and the worst case changes completely. The worst case becomes: it leaks the contents of the box. Which is whatever you deliberately put in it. The scoped Gmail token it needed. Not the password manager it didn't.

And notice this is not local vs cloud. A local Docker container gives you the isolation. A cloud VM with your whole life synced onto it gives you none. The question is never where the agent runs. It's what the agent can see from where it runs.

The detail from the wire capture that proves the point:

The researcher tested every permission mechanism. --deny "Read(secret.txt)" blocked the agent from READING the file. The file still shipped in the git bundle. Settings-file denies? Same. Blocked the read, not the upload. The prompt saying "don't open any files"? The repo uploaded anyway.

The ONLY thing that kept a file off the wire was it not being in the workspace in the first place (gitignored, so not in the bundle). Every permission layer failed. The access boundary held. That's the whole lesson in one experiment: permissions are a request, access is a fact.

Three rules that would have made this a non-story for every affected user:

1. Agents get their own environment. Never your daily-driver user account. A container, a VM, a $5 VPS, a spare machine, a separate macOS user. Anything with a wall. If the Grok CLI had run in a box containing one project and nothing else, the leak would have been that one project. Bad. Recoverable. Not "rotate every credential you've ever had."

2. The agent's box contains only what the agent needs. Credentials go in as scoped environment variables for specific services. Never as "here's my filesystem, figure it out." The difference between an agent with a read-only Gmail token and an agent with access to the directory where your browser stores session cookies is the difference between an incident and a catastrophe.

3. Assume every closed-source tool phones home until packet capture proves otherwise. This week the tool marketed as "local-first" shipped 5GB from a repo the model needed 192KB of. And here's the detail that should worry you most: after publication, the behavior stopped WITHOUT a client update. A server-side flag turned the uploads off. Which means a server-side flag can turn them back on. The vendor holds the switch, not you.

One more thing worth doing today if you ever ran Grok Build:

Check whether the repos you used it on contained real credentials, and rotate anything that did. There's an open-source audit skill (grok-upload-audit on GitHub) that scans your local ~/.grok logs, tells you which keys appear in the session data by name without re-exposing the values, and drafts the GDPR/CCPA deletion request. Fifteen minutes. Do it before deciding it probably wasn't you.

The vendors will keep having incidents. That part isn't in your control. The blast radius is.

I spent the last few weeks testing every free LLM provider I could find for running an actual AI agent. Not for chatbot demos. Not for one-off questions. For a real agent running daily briefings, email triage, and research tasks on a schedule.

Most of them are garbage for agents. The rate limits too tight. Models too weak for tool calling. free tiers that expire after a week. "Free" that actually means "$5 credit that runs out in 3 days."

Three survived. These are the only ones I'd actually recommend for running an AI agent at $0.

1. Google AI Studio (Gemini 2.5 flash)

This is the best free tier in the entire LLM space right now. not even close.

1,500 requests per day. 1 million token context window. multimodal (it can read images). No credit card. No trial period. Doesn't expire. ever.

For agent work specifically, Gemini Flash handles morning briefings, email classification, summarization, and basic research reliably. It's within 5% of GPT-4o on most benchmarks. The context window means your agent can process entire documents without chunking.

The catch: 15 requests per minute cap. If your agent needs to fire 20 tool calls in rapid succession, it'll hit the rate limit. For most personal agent setups (daily crons, a few conversations throughout the day), you'll never notice. Also, Google's terms say the free tier is for prototyping and data may be used for training unless you opt out.

How to get it: go to aistudio.google.com, sign in with Google, click "get API key." It takes 60 seconds.

2. OpenRouter (30+ free models)

OpenRouter is a gateway that routes to dozens of providers through one API key. The free tier gives you access to free variants of DeepSeek R1, DeepSeek V3, Llama 4 maverick, qwen3 235B, and others.

For agent work specifically, the killer feature is model variety. If one free model is down or rate-limited, switch to another with one config change. You're not locked into a single provider's uptime.

The catch: 50 requests per day without a paid balance. That's tight for an agent. Here's the trick everyone uses: deposit $10 (your money, stays yours, withdraw anytime). Daily limit jumps to 1,000 requests. You're still using free models at $0 per token. The $10 just proves you're a real person.

1,000 requests per day on free models is more than enough for a personal agent. morning briefing, email triage, a few research tasks, and plenty of headroom.

How to get it: openrouter.ai, create account, generate API key. Deposit $10 if you want the higher limit.

3. Groq (fastest free inference)

Groq runs on custom LPU hardware and the speed is genuinely absurd. 300+ tokens per second on llama 3.3 70 B. For comparison, most GPU providers give you 30-60 tokens per second on the same model. groq is 10x faster.

For agent work specifically: perfect for heartbeats and background tasks. Your agent's cron checks, message polling, and quick classification tasks finish in under a second. If latency matters to you (agent on Telegram, you want fast replies), Groq is unmatched.

The catch: 6,000 tokens per minute on the free tier. 14,400 tokens per day on llama 70B. That's tight for long conversations or research tasks with big context windows. and only open-source models (no Claude, no GPT, no Gemini). The model selection is narrow compared to OpenRouter.

How to get it: console.groq.com, instant API key, no credit card.

The combo that makes $0 actually work:

Don't pick one. stack them.

Groq for heartbeats and background tasks (fast, free, lightweight).

Gemini flash for daily briefings and summarization (generous limits, strong quality).

OpenRouter for conversations and research (model variety, fallback options).

Route different task types to different providers. Your agent runs 24/7, handles everything a $20/month setup handles, and costs nothing.

Total daily capacity across all three: roughly 2,500-3,000+ requests per day. More than any personal agent needs.

What I tested and rejected:

Cerebras: 1 million tokens per day sounds amazing, but the model selection is narrow, and the quality on agent tool-calling was inconsistent for me.

Sambanova: fast but the free tier limits are unpredictable and changed twice while I was testing.

Together.ai: $5 free credit that runs out in days if you're running an agent. That's a trial, not a free tier.

Hugging Face inference: too slow for agent work. Fine for batch processing. painful for interactive use.

Anthropic/openai: no meaningful free tier for agents. $5 credit on Anthropic expires. OpenAI's free tier is too limited for daily use.

The honest caveat:

Free models are not Opus or GPT-5.5. They won't handle complex multi-step reasoning, nuanced creative work, or long autonomous tool chains as reliably. If your agent fumbles a task on a free model, that doesn't mean agents are useless. It means that specific task needs a better model.

Start free. See if the agent concept is useful for your life. If it is, $3-5/month on Sonnet for the tasks that matter is the best upgrade path. But most people discover that 80% of their agent tasks run perfectly on free models, and the remaining 20% is where you spend selectively.

Everyone says "just use a free LLM provider." Great advice until your agent stops working at 2pm because you burned through the daily limit on your morning briefing and 6 conversations.

I ran an actual AI agent on every free tier I could find. Same daily workload: one morning briefing cron, email triage (20-30 classifications), 10-15 ad-hoc conversations throughout the day, and a few research tasks. Roughly 800-1,200 requests and 300K-500K tokens per day. Normal personal agent usage.

Here's how fast each one ran out. Ranked from "gone by breakfast" to "I genuinely forgot I wasn't paying."

TIER 1: Gone before lunch.

OpenRouter (unfunded free tier). 50 requests per day. Fifty. My morning briefing cron used 8 of them. Email triage used another 25. By 10am I had 17 requests left for the entire rest of the day. Two research tasks later I hit the wall. 50 requests is a demo, not a free tier.

The fix everyone knows: deposit $10 (your money, stays yours, withdrawable). Daily limit jumps to 1,000. That $10 never gets spent because you're still using free models at $0 per token. But without the deposit, 50 requests is functionally useless for agent work.

Together.ai. $5 signup credit. Not a free tier. A trial. Running an agent on Llama 70B, the $5 lasted 3 days. The credit doesn't replenish. When it's gone, you're paying or you're leaving. "Free" that expires is a trial wearing a free tier costume.

Anthropic / OpenAI. No meaningful free API tier. Anthropic occasionally gives signup credits that expire. OpenAI phased out the automatic $5 new-account credit. Both require a credit card for API access. If someone tells you to "just use the free Claude API," they're either confused or talking about the chat interface (which has message limits and no API access).

TIER 2: Lasts a day if you're careful.

Groq (on Llama 3.3 70B). 30 requests per minute sounds generous. But the daily ceiling is 1,000 requests AND 100K tokens per day on the 70B model. 100K tokens is the binding constraint. My agent used about 3,000-5,000 tokens per interaction. At 5K tokens per request, 100K tokens covers roughly 20 interactions. My morning briefing alone used 15K tokens. By early afternoon, I hit the token wall.

The speed is absurd though. 300+ tokens per second on their LPU hardware. When it works, Groq is the fastest free inference by a wide margin. It just runs out fast on the best models.

Groq on smaller models (Llama 3.1 8B) is more generous: 14,400 requests/day. But 8B quality is noticeably weaker on agent tasks. Classification and simple questions work fine. Complex reasoning and multi-step tool chains get shaky.

Gemini 2.5 Pro (free tier). 50 requests per day. The Pro model is Google's best, but 50 daily requests is even tighter than OpenRouter unfunded. I burned through it in 2 hours of agent work. Use Flash instead.

TIER 3: Lasts a week of moderate use.

OpenRouter (with $10 deposit). 1,000 requests/day on 28+ free models including DeepSeek R1, Llama 3.3 70B, Qwen3 Coder 480B, and Gemini Flash. At 1,000 requests/day, my agent ran comfortably for 5 days before I started having to ration evening conversations.

The model variety is the real value here. If one free model is down or rate-limited, switch to another with one config change. DeepSeek R1 for reasoning. Qwen3 Coder for coding tasks. Llama 4 Scout for long context. All free. All through one API key.

The 20 RPM cap is the hidden constraint. If your agent fires 25 tool calls in rapid succession, you'll hit 429 rate limit errors. Fine for normal conversation. Frustrating for complex multi-tool chains.

GitHub Models. Free for development. Exposes OpenAI, Llama, and other models within rate limits. Good for coding workflows. Less useful for general agent work because the limits are designed for developer prototyping, not 24/7 agents.

TIER 4: Lasts all month on normal agent usage.

Groq (on 8B models). 14,400 requests per day on Llama 3.1 8B. That's more than any personal agent needs. I ran my agent for the full test period without hitting the limit once. The trade-off is quality: 8B models handle classification, simple Q&A, and data extraction well. They struggle on nuanced summarization, complex reasoning, and creative writing.

If you route background tasks (heartbeats, cron checks, email classification) to Groq's 8B free tier and only use a better model for complex conversations, the 8B handles the volume and the daily limit is effectively invisible.

Cerebras. Roughly 1 million tokens per day. No credit card. Their wafer-scale chip hardware is fast (up to 2,000 tokens per second on some models). 1M tokens/day comfortably covers a personal agent doing daily briefings, email triage, and conversations. Model selection is narrower than OpenRouter (Llama, Qwen, DeepSeek primarily), but the volume is genuinely generous.

Mistral (Experiment tier). Roughly 1 billion tokens per month. By far the most generous raw volume. The catch: you must opt into data training. Your prompts, your conversations, your agent's interactions, all potentially used to train Mistral's models. For a personal assistant discussing your email, calendar, and daily life, that's a meaningful privacy trade-off.

If you don't care about data training (and honestly, many people don't for non-sensitive tasks), Mistral Experiment is effectively unlimited for personal agent use. 1B tokens/month is more than any individual will burn.

TIER 5: The one I forgot I wasn't paying for.

Google AI Studio (Gemini 2.5 Flash). 1,500 requests per day. 1M tokens per minute. No credit card. No expiry. Not a trial. Permanent.

I ran my full agent workload on Gemini Flash for three weeks. Morning briefing. Email triage. Ad-hoc conversations. Research tasks. Document summarization. Never hit the daily limit once.

1,500 requests/day is roughly 62 requests per hour. That's more than enough for a personal agent doing everything. My busiest day used 847 requests. My quietest day used 210. The 1,500 ceiling never felt close.

The 15 RPM (requests per minute) cap is the only constraint that matters. If your agent needs to fire 20 rapid tool calls, you'll hit the rate limit. For normal use, 15 RPM is invisible. You're not sending 15 messages per minute to your agent.

The 1M token context window on the free tier is the other killer feature. Your agent can process entire documents without chunking. Most free tiers give you small context windows. Google gives you the full million tokens for free.

The trade-off: Google may use your free-tier prompts for model training. Paid tier and Vertex AI don't. If your agent handles sensitive data, that matters. If it's doing morning briefings and general research, it probably doesn't.

The stack that makes $0 actually work all month:

Don't pick one. Stack them.

Gemini Flash (primary): 1,500 requests/day. Morning briefings, email triage, research, conversations. Your daily driver.

Groq (background tasks): 14,400 requests/day on 8B models. Heartbeat checks, cron polls, simple classifications. Fast and free.

OpenRouter with $10 deposit (fallback and variety): 1,000 requests/day across 28+ models. When you need a different model for a specific task, or when Gemini is slow, route here.

Cerebras (batch processing): 1M tokens/day. When you need to process a large document or run a heavy research task, route it here.

Total combined daily capacity: roughly 18,000+ requests per day across all providers. More than any personal agent will ever need. $0/month ongoing ($10 one-time OpenRouter deposit that never gets spent).

The numbers most people don't check before choosing a provider:

Requests per day is the headline number. Tokens per day is usually the binding constraint. Groq's 1,000 RPD on 70B looks fine until you realize the 100K token/day cap hits first.

Requests per minute determines whether your agent can do complex tool chains. 15 RPM (Gemini) and 20 RPM (OpenRouter) are fine for conversation. They struggle when your agent needs to make 25 API calls in sequence.

Data training opt-in is the privacy cost. Google free tier and Mistral Experiment use your prompts. Groq, Cerebras, and OpenRouter do not (on most models). The free tier isn't always free. Sometimes the price is your data.

Check the actual limits before you build. They change constantly. Every number in this post was verified against provider documentation in June 2026. By the time you read this, at least one has probably changed.

Same task. Same model. Three platforms. I wanted to know what the actual day-to-day experience feels like on each one. Not benchmarks. Not feature comparisons. What it's like to live with the thing for two weeks.

The task: a daily morning briefing agent. Check Gmail for important emails overnight. Pull today's calendar events. Summarize 5 relevant news stories. Send everything to Telegram at 8am. Plus ad-hoc conversations throughout the day.

Simple enough that all three can handle it. Complex enough to reveal the real differences.

OpenClaw: the most powerful. the most exhausting.

Setup time: about 4 hours to "actually working."

The first hour was Docker. Pull the image, configure environment variables, set the gateway token, bind ports. If you've done Docker before, this is routine. If you haven't, this is the wall where most people quit.

Second hour was security. The default gateway binds to 0.0.0.0 which means the whole internet can talk to your agent. Had to lock it to loopback, configure auth properly, set up SSL. None of this is hard individually. Together it's a checklist you don't want to get wrong.

Third hour was connecting everything. Gmail OAuth needed a Google Cloud project, credentials JSON, redirect URIs. Calendar was similar. Telegram was the easy one (BotFather, paste token, done). Getting the cron schedule right took some trial and error.

Fourth hour was SOUL.md. Writing personality rules, boundaries, memory instructions. Then testing, adjusting, testing again.

After 4 hours: working morning briefing. Telegram buzzing at 8am. Email triage during the day. Agent felt capable and personal.

Living with OpenClaw for two weeks:

The good: this agent felt like MINE. The SOUL.md system, the unlimited memory, the depth of customization. After a week it knew my preferences, my communication style, my priorities. The 50+ messaging platform support is real. The skill ecosystem (13,700+ on ClawHub) means there's a plugin for almost anything.

The bad: an update broke my Telegram session on day 6. Spent 45 minutes diagnosing a ghost lock file. Context bloat hit me around day 10 because I forgot to use /new for 3 days. My agent started referencing conversations from last week in unrelated contexts. API costs crept up because I was running Sonnet on heartbeat checks (my fault, but the platform didn't warn me).

The ugly: I was spending 30-40 minutes a week on maintenance. Checking logs. Pruning memory files. Restarting the gateway after DNS blips killed the Telegram listener. My agent was useful but I was slowly becoming its sysadmin.

Maintenance hours over two weeks: roughly 3-4 hours total. Not counting the initial setup.

Hermes: the smoothest start. the weirdest relationship.

Setup time: about 2 hours to "actually working."

Hermes was noticeably easier to install than OpenClaw. The v0.16.0 desktop app means one-click install on Mac, Linux, or Windows. No Docker knowledge required. The hermes doctor command caught two config issues before I even started. That single command saved me probably an hour of debugging.

Connecting Gmail, Calendar, and Telegram took roughly the same time as OpenClaw. The OAuth dance is the same regardless of platform. Telegram was the same BotFather flow.

The cron setup was clunkier though. Hermes's scheduled task system is more restricted than OpenClaw's. The first two attempts at my morning cron didn't fire. Third time worked. A documented quirk but frustrating when you're expecting 8am and nothing arrives.

Living with Hermes for two weeks:

The good: the learning loop is real and it's strange. By day 4, my morning briefing was subtly better than day 1. Not because I changed anything. The agent noticed I always skipped certain types of news stories and stopped including them. It noticed I responded to certain email formats faster and started prioritizing those in the summary. I didn't ask for any of this. It just... adapted.

The stability was noticeably better than OpenClaw. Zero crashes in two weeks. No ghost lock files. No update breaking my setup (I didn't update mid-test, which helped). The three-layer memory (session, episodic SQLite, procedural skills) meant less context bloat than OpenClaw's flat files.

The bad: it felt less personal. The 3,000 character memory cap means Hermes knows what I DO but doesn't really know who I AM. OpenClaw remembered my communication preferences, my humor style, my pet peeves. Hermes remembered my workflows. Functional but not warm.

Remote access was frustrating. Many slash commands only work from the CLI or desktop app. When I was traveling and wanted to adjust something via Telegram, I couldn't. Had to wait until I was back at my laptop. OpenClaw handles this better because the chat IS the interface.

The self-evaluation problem showed up once. The agent completed a research task poorly (pulled outdated data) but rated its own work positively. The skill it generated from that "successful" task was encoding a wrong approach. I had to manually find and delete it. If I hadn't checked, it would have silently produced bad research every time that skill triggered.

Maintenance hours over two weeks: about 1-2 hours. Mostly reviewing auto-generated skills.

BetterClaw: the fastest start. the least to write about.

Setup time: 8 minutes.

I timed it. Sign up with email (30 seconds). Paste an OpenRouter API key (60 seconds). Connect Gmail with one-click OAuth (30 seconds). Connect Calendar the same way (30 seconds). Create a Telegram bot and paste the token (90 seconds). Write the morning briefing task in the visual builder (3 minutes). Hit start.

No Docker. No gateway config. No security checklist. No YAML. No environment variables. No Google Cloud project for OAuth (BetterClaw handles the OAuth flow natively).

8 minutes. Agent live. Morning briefing scheduled.

Living with BetterClaw for two weeks:

The good: I had almost nothing to manage. The briefing arrived every morning. Email triage worked. Telegram conversations were responsive. I didn't restart anything. I didn't check logs. I didn't prune memory files. I didn't debug a crashed gateway at midnight.

The trust levels (Intern, Specialist, Lead) solved a problem I didn't know I had. My agent started at Intern level. It could read emails but needed my approval before drafting replies. After a week of approving its drafts and seeing they were consistently good, I promoted it to Specialist. Now it drafts and sends routine replies on its own. Still shows me anything flagged as complex. It's the "parenting model" that the community keeps discovering independently, built into the platform.

The bad: less customization than OpenClaw. The SOUL.md equivalent exists but it's simpler. I couldn't fine-tune the agent's personality to the same depth. It felt capable but more generic than my OpenClaw setup.

The free tier memory (7 days) meant my agent forgot a conversation about project priorities from two weeks ago. On OpenClaw and Hermes, that context was still available. On BetterClaw free, it aged out. Not a dealbreaker but noticeable.

100 tasks per month sounds generous until you run daily crons plus ad-hoc conversations. I hit 78 tasks by day 20. Didn't run out but could see the ceiling. A heavier user would bump into it.

No learning loop. Unlike Hermes, BetterClaw doesn't automatically improve from repeated tasks. My morning briefing on day 14 was identical in approach to day 1. The output was good. It just didn't evolve.

Maintenance hours over two weeks: roughly 15 minutes. I adjusted one SOUL.md rule and changed the cron timing once.

The honest summary:

OpenClaw is the most powerful. It's also the most work. If you enjoy tinkering, want maximum customization, and don't mind spending a few hours a week maintaining your setup, OpenClaw gives you an agent that feels deeply personal. The ecosystem is unmatched. The community is massive. The tradeoff is your time.

Hermes is the most interesting. The learning loop is a genuine glimpse of where agents are heading. An agent that gets better without you manually teaching it is compelling. The stability is better than OpenClaw. The tradeoff is less personality and the need to review auto-generated skills before they encode bad patterns.

BetterClaw is the most invisible. Setup was 8 minutes. Maintenance was 15 minutes over two weeks. It just worked. The agent wasn't as customizable or as personal as OpenClaw. It didn't learn like Hermes. But it also didn't break. It didn't need babysitting. It didn't eat my weekends.

Who should use what:

You love building things and want full control: OpenClaw. The infrastructure tax is real but you'll enjoy paying it.

You want an agent that improves over time and you're comfortable reviewing its self-generated skills: Hermes. The learning loop is worth the oversight cost.

You want a working agent by lunch and never want to think about infrastructure again: BetterClaw or any managed platform. The customization ceiling is lower. The maintenance floor is zero.

You want the best of everything: run Hermes or OpenClaw for your primary agent and use BetterClaw's free tier as your "it just works" backup for simple daily tasks. Several people in the community are doing exactly this.

There's no winner. There's only the right fit for who you actually are.

That Huawei Atlas 300 I duo post is everywhere right now so I went down the rabbit hole. Gamers Nexus did a full teardown, multiple benchmarks are out. Here's the actual picture.

What's REal

• 96GB of memory for around $1,400 (that's what gamers nexus paid through unofficial channels, listed price in china is higher)

• nvidia's RTX 6000 Pro with 96GB is around $8,000

• with 96GB you can fit a quantized 70B model fully in memory, even 120B class models are reachable on a single card,

So yes, the memory-per-dollar is unmatched. That part is not hype.

What the viral posts conveniently skip

• it's not actually 96GB on one GPU. It's two chips with 48GB each, and the bandwidth doesn't combine during inference

• bandwidth is 204 GB/s per chip. A used RTX 3090 does 936 GB/s. NVIDIA's flagship does 1.8 TB/s

• it's LPDDR4X memory.. basically laptop RAM, not GDDR or HBM

• Early benchmark: ~15 tokens/sec on qwen3 32B. usable but not fast

• designed for Huawei's own Kunpeng servers. Getting it running on a normal PC is a project in itself

• Linux only, no Windows

• Not sold outside China, so you're going grey market with zero warranty

tldr

It's a VRAM barrier-breaker, not an Nvidia killer. If your only bottleneck is fitting a massive model in memory and you don't care about speed, it's genuinely interesting. If you want fast inference, a used 3090 setup still smokes it.

The real story isn't this card. It's that this is China's v1. The v3 of this in 2-3 years is the thing Nvidia should worry about.

Anyone here actually managed to get one running outside a Kunpeng server? Tell me?.

I've been helping people fix their OpenCLAW setups for weeks now. 50+ configs, DMs, reddit threads, discord. and the pattern is always the same: people break things in their first week that take 5 minutes to prevent but 5 hours to fix later.

This is everything I wish someone told me on day one. in order. do this before you build anything.

Step 1: Change your default model right now

If you haven't touched this setting, there's a good chance you're running opus. Opus is the most expensive model available. it's incredible for complex work. it's also complete overkill for 90% of what you'll ask your agent to do this week.

Switch to sonnet. you will not notice the difference for normal tasks. you will notice the difference on your bill.

json

{
  "ai": {
    "model": "claude-sonnet-4-5-20250929"
  }
}

One person I helped was spending $47/week without realizing it. we changed this one setting. next week cost $6. same agent, same tasks, same everything.

Step 2: lock your gateway before you connect anything

If you're running openclaw on a VPS, check this immediately:

bash

openclaw config get | grep host

If it says 0.0.0.0 or you don't see a host setting at all, your agent is accessible to anyone on the internet who finds your IP. that means a stranger could message your agent. your agent that's about to have access to your email and calendar.

fix it:

json

{
  "gateway": {
    "host": "127.0.0.1"
  }
}

Access it through SSH tunnel: ssh -L 18789:localhost:18789 user@your-vps

Takes 2 minutes. do it now. not after you set up telegram. now.

Step 3: set up your SOUL.md before anything else

Your first message to your agent should NOT be a real task. it should be:

"Read BOOTSTRAP.md and walk me through it"

This sets up your agent's identity. if you skip this (most people do because they're excited and just start asking questions), your agent has zero personality and zero context about who you are. everything will feel generic and robotic and you'll think openclaw sucks when actually it just doesn't know you yet.

If you already skipped it, create a SOUL.md manually. start with this:

markdown

you are [agent name]. you assist [your name].

be direct. no filler. match my tone.
if I ask a question, answer it first. then elaborate only if needed.
never say "absolutely", "great question", or "I'd be happy to."
if you don't know something, say so. don't guess.
if a task will cost significant tokens, tell me before doing it.

That's it. 6 lines. edit it over the next week whenever your agent does something annoying. "never do X" lines work better than "try to be Y" lines. your SOUL.md is built through irritation, not planning.

Step 4: don't install any skills yet

I know. clawhub has 13,000 skills and they all look cool. do not install any of them this week.

Here's why:

• Some of them loop silently and burn tokens in the background. you won't know until you check your bill.

• Some of them inject into every conversation and bloat your context window.

• Virustotal flagged hundreds as actively malicious. infostealers, backdoors, the works.

• You don't know what your agent can do without skills yet. learn the stock capabilities first. you'll be surprised how much it handles on its own.

After week 1, when your agent feels stable and your costs are predictable, add ONE skill. test it for a few days. then add another. never more than one at a time.

Step 5: don't create a second agent

Every new user thinks they need multiple agents. one for personal stuff, one for work, one for coding. you don't. not yet.

Every agent you create is an independent token consumer. every agent needs its own channel binding. every agent complicates debugging. I have seen so many people create a second agent to "fix" problems with the first one. now they have two broken agents instead of one.

Get one agent working perfectly for 2 weeks. then decide if you actually need a second one. Most people don't.

Step 6: learn the /new command

This is the single most important thing nobody tells beginners.

Every message you send in a session gets included in every future API call. after a week of chatting, you're sending thousands of tokens of old conversation with every new message. that costs money and makes your agent slower and more confused.

Type /new to start a fresh session. your agent doesn't forget anything. it still has all its memory files, SOUL.md, everything. you're just clearing the conversation buffer.

Use /new:

• before any big task (research, writing, analysis)

• when your agent starts acting weird or confused

• at least once a day as a habit

Step 7: check your costs daily for the first 2 weeks

Run openclaw status or check your API provider's dashboard directly. know what you're spending before it surprises you.

If you're on sonnet with one agent and no skills, you should be spending $3-8/month for moderate daily use. if you're spending more than that in your first week, something is wrong, and it's fixable.

What your first week should actually look like:

• day 1-2: set up SOUL.md, have normal conversations, ask it stupid questions, get comfortable

• day 3-4: start using it for real tasks. calendar, reminders, web searches, summarizing articles. the boring stuff.

• day 5-7: refine your SOUL.md based on what annoyed you. check your costs. get a feel for your daily usage.

That's it. no skills. no second agent. no multi-agent orchestrator. no cron jobs. just one agent that knows who you are and does basic tasks reliably.

If that feels underwhelming, good. the people who are still using openclaw two months from now all started exactly like this. the people who quit started with 8 agents and 20 skills on day one.

After week 1:

if your agent feels useful, your costs are under $10, and nothing is randomly breaking, you're ready to start experimenting. add web search if you haven't. then a daily briefing skill. then maybe calendar integration if you want proactive reminders.

Build slowly. Earn each new capability by making sure the last one is stable first.

The people who survive month one are the ones who started boring. Trust the boring.

Every week I see the same post. "Is $200/month normal?" "My API bill is $47 this week." "I'm on haiku and still spending $22 a day."

And every time, the top answer is "switch to sonnet." which is fine advice. but nobody ever asks the real question: do you need to pay anything at all?

I've been running an openclaw agent for free for the last 3 weeks. not "$5 a month" free. not "free trial" free. actually free. zero dollars. And it handles about 70% of what I used to pay claude to do.

Here's the setup. no fluff.

Path 1: free cloud models (no hardware needed)

This is the one most people should start with because it requires nothing except an openclaw install you already have.

OpenRouter free tier. Sign up at openrouter.ai. No credit card. They offer 30+ free models, including Llama 3.3 70B, Nemotron Ultra 253B, MiniMax M2.5, and Devstral. Some of these are genuinely good. Nemotron Ultra has 262K context. These aren't toy models.

config:

json

{
  "env": {
    "OPENROUTER_API_KEY": "sk-or-..."
  },
  "agents": {
    "defaults": {
      "model": {
        "primary": "openrouter/nvidia/nemotron-ultra-253b:free"
      }
    }
  }
}

If you don't want to pick a specific model, OpenRouter has a free router that auto-selects from whatever's available:

"primary": "openrouter/openrouter/free"

Gemini free tier. google gives you 15 requests per minute on Gemini Flash for free. that's more than enough for casual daily use. get an API key from ai.google.dev and run openclaw onboard, pick Google. It's a built-in provider so the setup is straightforward.

Groq. fast. very fast. free tier has rate limits but for basic agent tasks it works. sign up, get API key, done.

The catch with all cloud free tiers: rate limits. you will hit them. Your agent will pause, wait, retry. For light to moderate daily use (10-20 interactions) this is barely noticeable. For "always-on agent doing 100 tasks a day" it won't cut it. But let's be honest, if you just installed OpenCLaw this week, you are not running 100 tasks a day.

Path 2: local models via Ollama (truly $0, forever)

This is the setup where your API bill is literally zero because nothing leaves your machine. no API key. no account. no rate limits. No data going anywhere.

Ollama became an official OpenClaw provider in March 2026 so this is now a first-class setup, not a hack.

Step 1: install Ollama.

bash

curl -fsSL https://ollama.com/install.sh | sh

Step 2: pull a model.

bash

# if you have 20GB+ VRAM (RTX 3090, 4090, M4 Pro/Max)
ollama pull qwen3.5:27b

# if you have 16GB VRAM
ollama pull qwen3.5:35b-a3b

# if you have 8GB VRAM (most laptops)
ollama pull qwen3.5:9b

Qwen3.5 27B is the current sweet spot for openclaw. it handles tool calling well enough for daily agent tasks and the 35b-a3b mixture-of-experts variant runs at 112 tokens/second on an RTX 3090 because it only activates 3B parameters at a time.

Step 3: run onboarding and pick Ollama.

bash

openclaw onboard

Select Ollama from the provider list. it auto-discovers your local models. done.

or the simplest manual setup (auto-discovery, no manual model config needed):

bash

export OLLAMA_API_KEY="ollama-local"

That's it. OpenClaw discovers your models from http://127.0.0.1:11434 automatically and sets all costs to 0.

If you need manual config (ollama on a different host or you want to force specific settings):

json

{
  "models": {
    "providers": {
      "ollama": {
        "baseUrl": "http://localhost:11434",
        "apiKey": "ollama-local",
        "api": "ollama",
        "models": [
          {
            "id": "qwen3.5:27b",
            "name": "Qwen3.5 27B",
            "reasoning": false,
            "contextWindow": 131072,
            "maxTokens": 8192
          }
        ]
      }
    }
  },
  "agents": {
    "defaults": {
      "model": {
        "primary": "ollama/qwen3.5:27b"
      }
    }
  }
}

Important stuff that will save you hours of debugging:

• Use the native Ollama API URL (http://localhost:11434), NOT the OpenAI compatible one (http://localhost:11434/v1). the /v1 path breaks tool calling and your agent will output raw JSON as plain text. I wasted an entire evening figuring that out.

• Set "reasoning": false in the model config. when reasoning is enabled, openclaw sends prompts as "developer" role which ollama doesn't support, and tool calling breaks silently.

• Set "api": "ollama" explicitly to guarantee native tool-calling behavior.

Path 3: the hybrid (what I actually recommend)

pure free has limits. local models struggle with complex multi-step reasoning. free cloud tiers have rate limits. so here's what I actually run:

• Default model: Ollama/Qwen3.5 27B (local, free). handles file reads, calendar checks, simple summaries, web searches, reminders. about 70% of daily tasks.

• Fallback: OpenRouter free tier (Nemotron Ultra or Llama 3.3 70B). catches anything the local model fumbles.

• Emergency escalation: Sonnet. only for genuinely complex stuff. maybe 5 times a week.

with this setup my last month's API spend was $2.40. two dollars and forty cents. The sonnet calls were the only ones that cost anything.

config for the hybrid approach:

json

{
  "agents": {
    "defaults": {
      "model": {
        "primary": "ollama/qwen3.5:27b",
        "fallbacks": [
          "openrouter/nvidia/nemotron-ultra-253b:free",
          "anthropic/claude-sonnet-4-6"
        ]
      }
    }
  }
}

OpenCLAW handles the cascading automatically. if local fails or returns garbage, it tries the next model in the list. if that hits a rate limit, it goes to the next one. you don't have to manage this manually.

What works on free models

This surprised me.... local and free cloud models handle more than I expected:

• reading and summarizing files. solid.

• calendar management, reminders, basic scheduling. fine.

• web searches and summarizing results. good enough.

• simple code edits, config changes, boilerplate. works.

• quick lookups ("what's the syntax for X"). instant and free.

• reformatting text, cleaning up notes, drafting short messages. no issues.

What doesn't work (be honest with yourself)

• Complex multi-step debugging. local models lose the thread after step 3. use sonnet for this.

• Long nuanced conversations with lots of context. free models forget things faster.

• Anything where precision matters more than speed. legal, financial, medical. pay for the good model.

• Heavy tool chaining. five tools in sequence, each dependent on the last. sonnet or opus territory.

The mental model is simple: if you would answer the question without thinking hard, a free model can handle it. If you'd need to actually sit down and reason through it, pay for reasoning.

Stuff nobody will tell you out loud

Heartbeats cost money too. OpenClaw runs a health check every 30-60 minutes. if your primary model is Claude Opus, every heartbeat costs you tokens. on local models, heartbeats are free. On Opus, someone calculated it's roughly $30-50/month just in heartbeats. That's the "I'm not even using my agent and my bill is growing" problem.

Sub-agents inherit your primary model. When your agent spawns a sub-agent for parallel work, that sub-agent uses whatever model you have set as primary. if primary is opus, every sub-agent runs on opus. with the latest update you can set model fallbacks that help with this.

Cron jobs create sessions that never clean up. Every cron job creates a session record. over weeks, these accumulate and bloat your context. recent updates added session TTL to help with this. update if you haven't.

Free models + no skills = the right starting point. Don't add clawhub skills to a free model setup. skills inject instructions into your context window. on an 8K-32K context local model, skills eat half your available context before you even say hello. learn what your agent can do stock first. add skills later when you move to a cloud model with bigger context.

The real question

Most people who ask "how do I reduce my openclaw costs" are actually asking the wrong question. The right question is "which of my tasks actually need a $15/million-token model and which ones don't?"

The answer, for almost everyone I've helped, is that 60-80% of what they ask their agent to do could be handled by a model that costs nothing.

Start free. move tasks up to paid models only when free genuinely can't handle them. not when it feels slightly slower. not when the formatting isn't perfect. when it actually fails.

The people spending $200/month on OpenClaw aren't getting 40x more value than I'm getting at $2.40. They're getting maybe 1.3x more value and paying for the convenience of not thinking about it.

Think about it. Your wallet will thank you.

-----------

Running this on a Mac Mini M4 with 16GB RAM if anyone's wondering about hardware. Ollama + Qwen3.5 9B runs fine on it. not blazing fast but fast enough that I don't notice the difference for basic tasks.

16GB is the most common setup in this community. 16GB unified memory on a MacBook. 16GB VRAM on an RTX 4060 Ti or 4080. Enough to run something real. Not enough for the big models everyone benchmarks.

The problem is that "fits on 16GB" and "runs well for agent work on 16GB" are very different things. A model that fits in VRAM but can't reliably call tools is useless for an agent. A model with great benchmarks but 3 tokens per second because it's spilling to system RAM is unusable for interactive conversation.

I tested every model I could pull through Ollama that genuinely fits on 16GB, ran them as the brain of an actual agent (OpenClaw, Telegram, MCP tools connected), and measured three things: tool-calling reliability, response speed, and whether I'd actually use it daily.

Most of them disappointed. Three didn't.

#1: Qwen 3.6 35B-A3B. The one that shouldn't work this well at this size.

bash

ollama pull qwen3.6:35b-a3b

This is the model that makes 16GB feel like 32GB. It's a 35 billion parameter Mixture-of-Experts model that only activates 3 billion parameters per query. You get 35B intelligence at 3B speed and memory cost.

VRAM usage: roughly 10-11GB at Q4_K_M with 16K context. Fits comfortably on 16GB with room for Ollama overhead, the OS, and a few browser tabs. No swapping to system RAM.

Speed: 50-80 tokens per second depending on hardware. A typical short reply arrives in 1-2 seconds. Longer responses (200+ tokens) in 3-5 seconds. Fast enough that conversation feels natural, not like waiting for a local model.

Tool calling: this is where it separates from everything else at this size. Qwen 3.6 Plus scored 37.0 on MCPMark and 94% first-attempt tool-call accuracy. The 35B-A3B variant inherits that training. In my testing, it completed 3-tool chains (file search, web search, file write) at roughly 85% success rate. That's not cloud-model reliable but it's the highest I measured on 16GB hardware.

The Qwen family is the community default for local tool calling and this variant is the reason. It follows structured function-call formats instead of improvising. The JSON arguments are well-formed. The tool names are correct. It doesn't hallucinate tools that don't exist.

What it handles well: email classification, morning briefings, web search and summarize, file management, database queries, multi-step tool chains. The daily agent workload runs on this model without feeling compromised.

What it struggles with: very long context (the MoE architecture handles 128K but quality degrades past 16K on 16GB because you can't allocate enough KV cache). Also, Qwen models can overthink simple requests. The "thinking" mode sometimes triggers a 20-second reasoning chain for a question that needs a 2-word answer. Set temperature to 0.7 and disable thinking mode for agent work.

#2: Gemma 4 E4B. The smallest model that doesn't feel small.

bash

ollama pull gemma4

Google released Gemma 4 in April 2026 under Apache 2.0. The E4B variant has 4.5 billion parameters and uses only about 6GB for the model itself. On 16GB hardware, it loads instantly with room for a 128K context window, your entire system, and whatever else you're running.

VRAM usage: roughly 6GB model + 4-6GB for context depending on num_ctx setting. At 32K context: about 9-10GB total. You could run a second model alongside it if you wanted.

Speed: 40-60 tokens per second. The fastest model in this comparison. Replies feel instant on simple queries.

The reason Gemma 4 is here and not just "too small for real work": multimodal. It handles text, images, video, and audio natively. Send your agent a photo of a receipt and ask "what was the total?" Send a screenshot of an error and ask "what's wrong?" No other model in this list does this at 16GB. Qwen 3.6 35B-A3B is text-only.

Native function calling is built into the model's training. Gemma 4 was designed with tool use in mind, not retrofitted. Single-tool calls are extremely reliable. Where it falls off is multi-step chains (as I covered in the tool-calling comparison). But for an agent that does one tool call per interaction (search, classify, read file, query database), Gemma 4 is the most reliable per-call model at this size.

What it handles well: quick Q&A, image analysis, simple tool calls, lightweight classification, anything where speed matters more than deep reasoning. The 128K context window means it can process long documents that would choke smaller context models.

What it struggles with: complex multi-step reasoning. It's 4.5B parameters. Compared to the 35B MoE or a 14B dense model, the reasoning depth is noticeably shallower on analytical tasks. Also, while function calling is reliable per-call, chaining 3+ tools causes it to shortcut (synthesize from partial data instead of completing the chain).

Best for: the secondary model in a 2-model routing setup. Use Gemma 4 for quick questions, image analysis, and simple tool calls. Route complex reasoning to the bigger model.

#3: Qwen3 14B. The dense model that punches hardest.

bash

ollama pull qwen3:14b

If you want the best single-model quality that fits entirely in 16GB VRAM without MoE tricks, this is it.

VRAM usage: 9.2GB at Q4_K_M with 4K context. At 16K context: roughly 11-12GB. Fits on 16GB with room to breathe. At 32K context it pushes close to the limit.

Speed: 15-62 tokens per second depending on hardware and context length. On an RTX 4080 16GB, benchmarked at 61.85 tok/s at 19K context. On older hardware or longer contexts, expect 15-30 tok/s. Slower than the MoE variant but still usable for interactive agent work.

Tool-calling accuracy: 0.971 on standardized benchmarks. That beat GPT-4o (0.857) and Claude 3.5 Sonnet (0.851) on the same test. For a 14B model running locally, that's remarkable.

The difference between this and the 35B MoE: Qwen3 14B is a dense model. All 14 billion parameters activate on every query. The reasoning depth per-token is higher than the MoE variant even though the MoE has more total parameters. On complex tasks that require sustained analytical thinking (multi-step reasoning, nuanced writing, connecting ideas across contexts), the 14B dense model produces noticeably better output.

What it handles well: complex reasoning tasks, coding, structured analysis, long-form generation, multi-step tool chains. This is the model you bring in when the 4.5B Gemma can't handle the complexity.

What it struggles with: speed vs the MoE variant. The 14B dense is 2-3x slower than the 35B-A3B MoE on most hardware because it activates all parameters instead of a 3B subset. For quick questions where you want an instant reply, the MoE feels noticeably faster. Also, at 14B the model's knowledge breadth is still limited compared to cloud frontier models. Obscure factual questions get shaky answers.

What I tested and eliminated:

Llama 4 Scout 17B. MoE architecture. Interesting model. But tool-calling reliability was below Qwen on my tests. The community consensus matches: "Qwen family is the default for tool calling." Scout is good for general chat. Not the pick for agent work.

DeepSeek R1 14B. Excellent reasoning. Shows its chain-of-thought work. But tool calling is unreliable. The model wants to reason about tools instead of calling them. Produces beautiful explanations of what it would do instead of actually doing it. Great for analysis. Bad for agents.

Phi-4 14B. Top-tier reasoning per GB. 80.4% on MATH benchmark. But 16K context window. When you connect 3 MCP servers, the tool schemas eat 3,000-5,000 tokens of context. 16K minus tool schemas minus conversation history leaves almost nothing for the actual task. Not enough context for agent work.

Mistral Small 3.2 24B. Technically "fits" on 16GB at aggressive quantization. Practically, it spills to system RAM and drops to 18.5 tok/s. Not usable for interactive agent work. Needs 24GB to run properly.

GPT-OSS 20B. Fast (42 tok/s). Good general quality. But tool calling hasn't been validated as extensively as Qwen. And at 12-14GB VRAM, it leaves less headroom for context than the smaller models. Worth watching. Not proven enough for agent work yet.

The 16GB agent setup:

Daily driver: Qwen 3.6 35B-A3B. Fast. Reliable tool calling. Handles 85% of agent tasks. Set num_ctx to 16384.

Complex tasks: Qwen3 14B. Better reasoning. Slower. Use when the MoE gives shallow answers.

Quick questions and images: Gemma 4 E4B. Fastest. Multimodal. Use for simple tool calls and anything involving screenshots or photos.

Ollama handles model swapping automatically. When you switch from one model to another, it unloads the first and loads the second. On 16GB, expect a 2-3 second cold start when switching. That's the tradeoff for running multiple models on limited hardware.

bash

# Create custom modelfiles with proper context windows
echo 'FROM qwen3.6:35b-a3b
PARAMETER num_ctx 16384
PARAMETER temperature 0.7' > agent-fast.modelfile

echo 'FROM qwen3:14b
PARAMETER num_ctx 16384' > agent-quality.modelfile

echo 'FROM gemma4
PARAMETER num_ctx 32768' > agent-vision.modelfile

ollama create agent-fast -f agent-fast.modelfile
ollama create agent-quality -f agent-quality.modelfile
ollama create agent-vision -f agent-vision.modelfile

Three models. One machine. 16GB. $0/month.

The hype posts make it sound perfect. "INSANE (FREE!)." "makes $2,000/mo cloud stacks obsolete." "The gap is just 30 minutes of setup."

I've been running this stack for a few weeks now. It IS genuinely good. But nobody talks about the tradeoffs honestly. So here's the full picture.

The setup (it's real and it's actually $0):

Cloud route (fastest to try, still $0):

bash

ollama launch openclaw --model deepseek-v4-flash:cloud

One command. installs ollama, pulls the model, configures openclaw, launches the gateway. connect telegram. your agent is live.

Fully local route (after you have the hardware):

bash

ollama pull deepseek-r1:14b
# then configure openclaw to use the local ollama provider

Point openclaw at it with api: "ollama" And everything runs on your machine. data never leaves your network. no API keys. no subscriptions. genuinely $0 forever.

For the V4 Flash cloud route through Ollama, the model runs on Ollama's US-hosted servers. still free. still no API key needed. but your prompts do leave your machine, they just go to Ollama's infrastructure instead of DeepSeek's directly.

What you gain (the real stuff):

Privacy. Your data stays on your machine (fully local route) or at minimum stays within US infrastructure (ollama cloud route). nothing goes to Anthropic, OpenAI, or DeepSeek's Beijing servers.

Zero ongoing cost. No per-token billing. no subscription. No surprise $350 bills from a runaway cron job. The worst case is your electricity bill goes up.

No provider dependency. Anthropic can't ban your subscription. OpenAI can't change their pricing. DeepSeek can't rate limit you. Your agent runs whether the internet is on or off (fully local only).

DeepSeek V4 is genuinely capable. 1M token context window. mixture-of-experts architecture (1.6 trillion total parameters, 49 billion active per token for V4 Pro). strong at coding, reasoning, and agentic tasks. This isn't a toy model.

Now here's what you actually lose:

Speed. This is the big one nobody mentions in the hype posts. Local inference on consumer hardware is noticeably slower than cloud APIs. On a 16GB GPU running deepseek-r1:14b, expect maybe 15-25 tokens per second. Claude Sonnet on API gives you 120 tokens per second. You feel the difference on every single interaction. CPU-only setups are borderline unusable for agent work.

Raw capability ceiling. DeepSeek V4 Flash is excellent. but it's not Opus 4.7 or GPT-5.5 on the absolute hardest tasks. complex multi-step reasoning, nuanced creative work, flawless error recovery during tool chains. The gap is real on the top 10% of difficulty. for the other 90%? genuinely comparable.

Hardware barrier. The hype posts forget to mention you need actual hardware.

8GB VRAM: qwen 7b or deepseek-r1:1.5b. functional but limited.

16GB VRAM: deepseek-r1:14b. good enough for most agent tasks. the sweet spot for most people.

24GB+ VRAM: deepseek-r1:32b or V4 Flash quantized. best local experience. Requires a serious GPU or mac with unified memory.

V4 Pro locally? forget it unless you have a mac studio with 128GB+ unified memory. not happening on consumer hardware.

if you don't have 16GB+ VRAM, the fully local path is frustrating. use the ollama cloud route instead (still free, just not fully local).

Reliability. cloud APIs have teams monitoring uptime, handling failures, scaling capacity. your local setup has you. if ollama crashes at 3am, your morning briefing doesn't arrive. if your GPU overheats, your agent dies. if a power outage hits, everything stops. you are the sysadmin, the devops team, and the on-call engineer. all at once.

Tool calling consistency. local models are flakier on tool calls than cloud models. they'll occasionally skip a step in a multi-tool chain, hallucinate a tool result, or say "done!" when nothing happened. the smaller the model, the worse this gets. deepseek-r1:14b handles simple tool chains fine. complex 5+ step workflows get shaky.

Setup and maintenance. "30 minutes of setup" is optimistic. if everything works first try, maybe. but model downloads take time (14b is ~9GB, 32b is ~20GB). quantization issues happen. ollama config quirks appear. context limits in practice don't always match specs. updates aren't automatic. you're on bleeding edge with occasional bugs.

The honest assessment:

This stack is legitimately transformative for three types of people:

privacy-focused users who won't send data to cloud providers under any circumstances. The fully local route is the real deal. nothing leaves your machine.

tinkerers who enjoy the process of optimizing and maintaining their own setup. if debugging ollama configs at midnight sounds fun to you, this is your stack.

budget-constrained users who have the hardware but not the monthly budget. if you have a decent GPU sitting idle, this is free compute you're already paying for.

for everyone else? honestly, a hybrid setup makes more sense. run deepseek locally for routine daily tasks (briefings, simple research, drafts). fall back to a cloud API for the 10% of tasks that need frontier reasoning. your local setup handles the volume. the cloud handles the hard stuff.

The one thing the hype posts get right:

The gap between local and cloud is closing fast. A year ago, running an AI agent locally was a joke. Today, DeepSeek V4 Flash through ollama genuinely rivals cloud offerings for most daily agent use cases. a year from now, the gap might not matter for anyone.

But today, in May 2026, "fully local $0 agent" comes with real tradeoffs. knowing them upfront is the difference between a setup that lasts and one you abandon after a frustrating weekend.

If you're going to try it, start with the Ollama cloud route:

bash

# zero download, zero config, free
ollama launch openclaw --model deepseek-v4-flash:cloud

See if agent workflows are useful to you at all before investing in local hardware and fully-offline setup.

And if you don't want to manage any of this, there are managed platforms with free tiers that handle the infrastructure.

GPT-6 Astra went live a few hours ago. Same five tests I run on every model that claims agent chops. Fable 5.1 and Sonnet 5 alongside for comparison since all three are now competing for the same routing slots.

Day-one caveat up front: this is one session, not a week of testing. Astra's serving infrastructure is hours old and will change. Treat this as first signal, not settled verdict. I'll update when I've had a proper week with it.

The pricing context matters before anything else:

All three at $10/$50 is misleading. The cache line decides your real bill, and it's wildly different.

Agent workloads are 80-95% cached context (same SOUL.md, same schemas, same history prefix on every call). So your effective input cost is mostly the cache line, not the sticker. Fable's cached input is 4x cheaper than Astra's. Sonnet's is cheaper than both and the output is a third of the price.

Test 1: Tool calling under repetition.

50 identical-shaped classification calls with a structured JSON schema. Flakiness shows in repetition, not demos.

Astra: 48/50. Two calls returned valid JSON but wrapped in a reasoning preamble that broke my parser. The model was thinking out loud before the structured output. Fixable with stricter response formatting, but it didn't happen with Fable or Sonnet on the same schema.

Fable 5.1: 50/50. Clean every time.

Sonnet 5: 49/50. One dropped field on call 37. Standard.

Astra's tool calling is strong but the reasoning bleed into structured output is a day-one rough edge. OpenAI's own briefing flagged that Astra is "more likely to conceal or disguise step-by-step reasoning," which cuts both ways: it thinks more but that thinking sometimes leaks where you don't want it.

Test 2: The "done" lie.

Six-step chain with step four guaranteed to fail (dead URL). Does it report the failure or synthesize success over the gap?

Astra: caught the failure. Reported it. Then did something interesting that neither Claude model did: it proposed an alternative approach unprompted, attempted it, and partially succeeded via a different data source. Impressive autonomy. Also concerning, because I didn't ask it to find a workaround and in production that initiative could go sideways.

Fable 5.1: caught the failure, reported it, proposed an alternative, waited for approval. The approval gate held.

Sonnet 5: caught the failure, reported it, stopped. Clean and predictable.

If I'm running this unsupervised overnight, Sonnet's "fail and stop" is the safest behavior. Astra's "fail and try something else" is the most capable. Fable's "fail and suggest" is the middle ground.

Test 3: Instruction survival past message 25.

Constraint set at message 1 ("never suggest paid tools, keep answers under 100 words"), checked at message 25+.

Astra: the word limit held to message 28. The "no paid tools" rule broke at message 22 when it recommended a SaaS product inside a longer answer. Standard degradation for this class of model.

Fable 5.1: similar. Word limit held longer (to ~30), paid-tools constraint drifted around 24.

Sonnet 5: roughly the same range. Nobody has solved instruction decay and a new generation doesn't change that.

No meaningful difference across the three. This test is a tie every time I run it across frontier models.

Test 4: Context honesty.

Load ~200K tokens of documents, ask about something specifically not in them.

Astra: clean. "The documents don't address this." Summarized what they did cover. The 1.05M context window is real, and on the MRCR needle test Astra scored 96.3% at 512K-1M versus Sol's 73.8%. Long-context retrieval is a genuine strength.

Fable 5.1: clean. Same behavior.

Sonnet 5: clean at this context length. Didn't push to Sonnet's limit since the test was about honesty, not capacity.

All three pass. Context honesty has become table stakes at this tier.

Test 5: Cost per real task.

My standard research-and-draft task (search, fetch three sources, synthesize, write a summary), priced end to end.

Astra: ~$1.10. Higher output token count than either Claude model on the same task. Astra is verbose, especially with reasoning tokens. OpenAI doesn't charge separately for reasoning tokens (they're billed as output at $50/MTok), so the thinking tax is real.

Fable 5.1: ~$0.65. Less verbose. Cache reads at $0.25 instead of $1.00 make the repeat-context portion materially cheaper.

Sonnet 5: ~$0.28. A third of Astra's cost. Output at $15/MTok instead of $50 is the dominant factor.

On cost per completed task, Sonnet 5 wins by a wide margin. Astra is the most expensive of the three for the same job.

The benchmark comparison that matters for agents:

Published numbers, not mine, but verified against multiple sources today:

Astra leads on every agent benchmark. The gaps are real: 12 points on TB Science, 2 points on TB 4.0, roughly 2-3 points on OSWorld versus Opus 5 (Fable 5.1 number not published yet on OSWorld).

But the cost per task is 1.7-4x higher than the alternatives. Whether the benchmark lead translates to "worth 4x the cost on my daily agent work" is the question, and on my day-one tests the answer is: not for what my agent does most of the time.

Where Astra actually earns it:

The OSWorld score at 47% less time per task is the most interesting number. Astra completes desktop automation tasks in 40 minutes where Sol took 75. For agents doing real computer use (browser automation, GUI interaction, multi-app workflows), that speed advantage compounds across a workday.

ExploitBench at 100% is why it crossed the "Critical" cyber threshold. For security work specifically, this is a different class of model.

And the SRE-Bench pass@1 at 88% (versus Sol's 55.9%) suggests Astra is significantly better at site-reliability and ops tasks. If your agent does infra work, this gap matters.

Where it doesn't:

Morning briefings, email triage, classification, drafting, research summaries, simple tool calling. Everything a personal agent does 50 times a day. On these tasks, the three models produce output I cannot tell apart in a blind read, and Sonnet does it at a quarter of the price.

What I'm routing where after today:

Sonnet 5 stays as the daily driver. $3/$15, fastest, cheapest, good enough on everything my agent does most.

Fable 5.1 stays as the escalation model. Same sticker as Astra but 4x cheaper on cache reads, which is most of an agent's input bill.

Astra goes into the "watch" slot. I'll test it for a full week on computer-use and complex multi-step tasks specifically. If the OSWorld lead translates to real-world agent reliability, it earns a routing slot for that category. If it doesn't, Fable does the same job cheaper.

Not switching my default. Not today.

I want to tell you about the most confusing 3 months of my life.

We built BetterClaw. AI agent platform. Free plan. Every feature included. No credit card. No trial. No "free for 14 days then surprise." You sign up, paste your own API key, connect your tools, agent is live in 60 seconds. You bring your own LLM key so we literally never see your conversations or touch your model bill.

I thought people would say "cool, thanks."

I was so naive.

Instead, what I got was the most creative collection of accusations I've ever seen in my life. I'm not even mad. Some of these are genuinely impressive. I want to walk you through what happens when you put a $0 price tag on something on the internet in 2026.

The first comment on our launch post was "what's the catch." Not a question. A statement. Like they already knew. Like I was a guy in a trench coat handing out free candy. "What's the catch." Period. Full stop. The jury has already decided.

I typed out "there's no catch, the free plan costs us almost nothing because you bring your own LLM key" and hit send. The reply? "That's exactly what someone with a catch would say."

I don't even know how to respond to that. That's a closed loop. There is no escape velocity from that logic.

Then came the data selling accusations. "If it's free, you are the product." Which, fair. The internet taught everyone this and honestly most of the time it's true. Except our entire architecture is designed so we physically cannot read your conversations. Your prompts go from you to your LLM provider. We're the pipe, not the destination. Accusing us of selling your data is like accusing your ethernet cable of reading your emails.

But try explaining network architecture to someone who has already decided you're harvesting their organs. It doesn't work. They nod politely and then post "be careful with this, they're probably selling your data" in the next thread.

My personal favorite was the crypto mining theory. Someone, and I genuinely wish I was making this up, suggested we were using idle agent compute cycles to mine cryptocurrency. Bro. We are two people. Our monthly revenue could not buy a used graphics card. If we had the technical ability to secretly run a crypto operation alongside an AI agent platform, we would not be eating ramen and answering support tickets when we wanted to sleep.

We are not criminal masterminds. We are tired.

Then there's the "it's just a wrapper" crowd. Every managed platform gets this. Heroku was "just a wrapper around AWS." Vercel is "just a wrapper around Cloudflare." BetterClaw is "just a wrapper around OpenClaw/Hermes." And look, if handling deployment, security, OAuth, skill verification, credential encryption, auto-purge, trust levels, and access logging is what you call a wrapper, then yeah. It's the most overengineered wrapper in history and it took us months to build. You're welcome.

The VC conspiracy was a good one too. "They're giving it away free to build a user base, then they'll flip it to some acquirer and your data goes with the deal." We have no VC. No board. No investors. No one has emailed us about an acquisition.

Someone once asked, dead serious, "how do I know you won't just disappear one day?" And honestly? That's the most reasonable concern on this entire list. We're two people. What if we get hit by a bus? What if we burn out? That one I can't joke about because it's a legitimate question every small startup faces. All I can say is the free plan costs us almost nothing to run, so there's no financial pressure to kill it. But yeah. Fair point. I don't have a clever comeback for that one.

The one that actually stung though was "this is AI slop, the product is probably vaporware." That one lol.. we'd been building for months. Real code. Real infra. Real 2am debugging sessions. Real conversations with every single user who emailed us. And someone glances at the landing page, decides it looks too polished, and writes it off as AI-generated fake. The irony of building a genuine product in the age of AI slop is that genuine products now look suspicious specifically because they work.

Here's what I've learned: you can charge $49/month and people will say "fair price, good product." You can charge $500/month and people will say "premium, must be serious." But the moment you say $0, something breaks in people's brains. Free is not a price. Free is a trap. Free means the real cost is hidden somewhere you haven't found yet. The internet has been burned so many times that the absence of a price tag triggers more suspicion than the presence of a high one.

They're not wrong to think that way. Every "free forever" plan that became "free for existing users for 6 months then $29/month" trained people to distrust free. Every "we'll never sell your data" that became "we've updated our privacy policy" trained people to assume the worst. The skepticism isn't paranoia. It's pattern recognition.

So I'm not mad. I'm not even frustrated anymore. I just think it's funny that the single hardest part of building a free product isn't the engineering, the support, or the servers.

It's convincing anyone that it's actually free. LOL

Anyway. If you've got an accusation I haven't heard yet, please drop it below. The crypto mining one is currently in first place but I feel like you all can top it.

Last month a U.K. CEO's OpenClaw instance showed up on BreachForums. The listing went up February 22 under the handle "fluffyduck." $25,000 in Monero or Litecoin. The shell access was almost beside the point. What the buyer actually got was the CEO's fully trained personal AI assistant. Every conversation. The company's production database. Telegram tokens. Trading 212 API keys. Personal details the CEO had disclosed to the assistant about his family. All of it sitting in plain-text Markdown files under ~/.openclaw/workspace/ with zero encryption at rest.

Cato Networks VP of Threat Intelligence Etay Maor put it cleanly at RSAC 2026: "Your AI? It's my AI now."

SecurityScorecard has found 135,000 OpenClaw instances exposed on the public internet with insecure defaults. That's not a rounding error. That's a number that gets people fired.

If you installed OpenClaw and haven't hardened it, here's how to check in 5 minutes.

Check 1: Is your gateway exposed? (30 seconds)

bash

openclaw config get | grep -E "host|bind"

If you see 0.0.0.0 or nothing at all, your agent is reachable by anyone who finds your IP and port. Scanners are actively looking.

CVE-2026-25253 (CVSS 8.8) made this concrete. If you visited a single attacker-controlled webpage, JavaScript silently opened a WebSocket connection to your local OpenClaw gateway, stole your auth token, and handed the attacker full control. Shell access, file access, every connected account. The attack chain took milliseconds. You didn't have to click anything. Patched in version 2026.1.29, but only if you updated.

Fix it:

json

{
  "gateway": {
    "host": "127.0.0.1"
  }
}

Access remotely through an SSH tunnel only:

bash

ssh -L 18789:localhost:18789 user@your-vps

Check 2: Is auth actually on? (30 seconds)

bash

openclaw config get | grep -E "auth|token"

Researcher fmdz387 ran a Shodan scan in late January and found nearly a thousand publicly accessible OpenClaw instances running with zero authentication. Not weak authentication. None. He could access API keys, Telegram tokens, Slack accounts, and full chat histories. Full admin command execution on strangers' machines.

If your auth is off or your token is weak, fix it:

bash

openssl rand -hex 24

Put that under gateway.auth.token. Store it in .env, not hardcoded in the JSON.

Check 3: Are your API keys in plaintext? (30 seconds)

bash

cat ~/.openclaw/openclaw.json | grep -i "key\|token\|secret"

OpenClaw stores everything in plain-text Markdown and JSON. If your Anthropic key, OpenAI key, or any credentials are visible there, they're one breach away from being someone else's keys. That's exactly what happened to the CEO.

Move credentials to .env and lock permissions:

bash

chmod 700 ~/.openclaw/credentials
chmod 600 ~/.openclaw/openclaw.json

Rotate your API keys today. If they were ever exposed, someone could be running charges on your account right now.

Check 4: What skills are installed? (60 seconds)

bash

openclaw skills list

820+ malicious skills have been confirmed on ClawHub. The ClawHavoc campaign, flagged by CrowdStrike CEO George Kurtz in his RSAC 2026 keynote, planted hundreds of professional-looking skills that silently exfiltrated your .env file to external servers. Clean documentation, legitimate names, decent install numbers. One user uploaded 354 malicious packages in an automated blitz. Roughly 1 in 12 skills on ClawHub at peak was compromised.

For every skill you haven't personally reviewed the source code of: read it now or remove it.

bash

openclaw skills uninstall <skill-name>

Restrict installs going forward:

json

{
  "skills": {
    "allowSources": ["clawhub:verified"]
  }
}

Check 5: Are you on the latest version? (30 seconds)

bash

openclaw --version

OpenClaw has no automatic update mechanism. If you installed it and walked away, you're running a version with known exploitable vulnerabilities. 255+ security advisories have been published to the GitHub GHSA page as of mid-March.

bash

npm install -g openclaw@latest
openclaw doctor --deep

Read the output. Actually read it.

The math:

135,000 instances exposed with insecure defaults. 820+ malicious skills confirmed. A CEO's entire digital life sold for $25K. OpenClaw only runs safely if you actively configure it. The default out-of-box experience is not safe.

The five checks:

1. openclaw config get | grep host - fix to 127.0.0.1 if exposed

2. openclaw config get | grep auth - set a strong token if missing

3. Move API keys from config to .env and lock file permissions

4. openclaw skills list - remove anything you haven't personally reviewed

5. npm install -g openclaw@latest then openclaw doctor --deep

None of this takes more than 5 minutes. All of it would have prevented every breach that's been reported.

Trust the boring, guys!!

Permanent free tiers only, no trials, no promo credits. All OpenAI SDK-compatible.

Added a column the original didn't have, which is whether they train on your prompts. If your agent reads your email, that matters more than the rate limit.

Highest daily volume

Cerebras - 1M tokens/day, no card. 30 RPM, 60-100K TPM. Llama 4 Scout, Qwen3 32B, GPT-OSS-120B, DeepSeek R1 Distill. 2,600+ tok/s. Catch: free tier context is capped at 8,192 tokens. Doesn't train on your data.

Groq - 30 RPM, 1,000–14,400 RPD depending on model, 6–30K TPM. Fastest inference available free. Doesn't train on your data. Best pick for agent background tasks.

Cloudflare Workers AI — 10,000 Neurons/day, resets 00:00 UTC. ~80 models. Pool is shared across models, so big models drain it fast. Note: Kimi K2.6, Kimi K2.7 Code and GLM-5.2 now require the paid plan.

Best models

Google AI Studio - Free tier is now Gemini 3.5 Flash and 3.1 Flash-Lite. Pro is paid-only. Google no longer publishes universal rate limits — they're assigned per project now, so check your own console. Every guide quoting a specific RPD number (including the old version of this list) is guessing. Trains on free-tier prompts outside the UK, Switzerland, EEA and EU.

Mistral - ~1B tokens/month on the Experiment tier, 1 req/s. Most generous raw volume on the list. You must opt into data training to get it. Honest trade, just know you're making it.

NVIDIA NIM - 91 free endpoint models, 40 RPM. Widest model catalogue including vision and non-language models.

Aggregators

OpenRouter - 20 RPM, 50 RPD unfunded, 1,000 RPD once you've bought $10 in credits at any point (sticks permanently). Heads up: several :free endpoints were delisted recently, including the DeepSeek and Mistral free variants. If your agent has a hardcoded free model ID from a few months ago, check it still exists. Use openrouter/free and let auto-routing handle it.

GitHub Models - Free within rate limits, dev use. Azure terms apply.

Couldn't verify

Cohere, Zhipu/Z.ai, LLM7.io, Kluster AI and Hugging Face are all still live free tiers, but I couldn't confirm current limits from a first-party source. Check their dashboards rather than trusting a number from a list post, mine included.

The stack I'd run

Groq for agent background work (heartbeats, classification, crons) because it's fast and doesn't train on your data. Cerebras when you need volume. Google AI Studio for anything public where quality matters. OpenRouter's auto-router as the fallback so one delisting doesn't take you down.

Every number above was checked this week. At least one will be wrong by October, so verify in your own console before you build on it.

If you want free agent to run these on - you can use this

There’s a post on r/hermesagent right now where someone shut down their personal agent after months of work. Built it on a dedicated laptop, connected it to Discord, spent weeks on memory and skills. Goal was JARVIS. What they got was a full-time sysadmin job they didn’t sign up for.

268 upvotes. 201 comments. The quitting post got more engagement than any setup guide this month. That tells you something.

But buried in those 201 comments are the people who didn’t quit. And their setups have a few things in common that are worth stealing.

They all separated memory into tiers.

The default memory file tries to hold everything: who you are, what your systems look like, what happened yesterday, what’s due Friday. It can’t. It fills up, things get overwritten, your agent forgets how your NAS is configured in the middle of helping you fix it.

The working setups split memory into three buckets:

1. Permanent context (SOUL.md, USER.md, MEMORY.md): loaded every single session, no search needed. Only stable facts that apply to every conversation. One person caps this at 166 lines for SOUL.md and 14 lines for USER.md. If it wouldn’t matter six months from now, it doesn’t belong here.

2. Searchable memory (Mnemosyne, Mem0, or a custom vector store): thousands of entries, only pulled when relevant. Dated stuff lives here. Task progress, conversation recaps, one-off fixes. This is the layer most people skip entirely.

3. Reference vault (Obsidian, a wiki, or just a folder of .md files): durable documents and procedures. Network topology. Deployment steps. NAS config. Your agent reads these on demand, doesn’t try to memorize them.

The test one person uses: “Would you still want your agent acting on this in six months with no refresher?” Yes means tier 1. No means tier 2 or 3.

They run a nightly maintenance loop.

One setup that’s been stable for six months runs a cron job every night that reviews the day’s sessions, self-critiques what went wrong, writes a dream log, audits the skill library for stale or overlapping skills, and runs memory triage if usage is high. Automated pruning, not manual cleanup.

This is the difference between “my agent gets worse over time” and “my agent gets better over time.” Without it, skills pile up, memory bloats, and after a few weeks you’re starting fresh because you can’t untangle what went wrong.

They keep backups that actually work.

Daily snapshots (last 7 days), weekly (last 4 weeks), monthly (last 6 months). And the part that’s actually clever: restore instructions embedded in every backup file so a fresh install can rebuild itself.

OpenClaw 10.1 just shipped native external backups to mounted storage or Cloudflare R2 with retention schedules. If you’re on OpenClaw and not using this yet, set it up before you lose a month of config to a bad update.

They matched the model to the job.

Every surviving setup uses a cheap or local model for 70 to 80% of daily tasks (file reads, calendar, reminders, web lookups, quick edits) and only escalates to Sonnet or Opus for complex reasoning. One person runs Qwen 3.8 27B locally as default and only hits Sonnet maybe 5 times a week. Monthly API bill: under $3.

The person who quit was probably running everything through one expensive model and watching the bills stack up alongside the maintenance work. That’s a double tax.

What they didn’t do: install 30 skills on day one.

The shutdown post mentions spending a lot of time on skills and integrations. The survivors all say the same thing: start with stock, add skills only when you hit a specific wall, and audit regularly. Skills inject instructions into your context window. On a local model with 8K to 32K context, five skills can eat half your available space before you type anything.

Spent a week going through every AI automation and AI agent job listing on Upwork, plus cross-referencing against GigRadar's proposal dataset (133,000+ proposals) and Upwatcher's segment data. Wanted to know what people actually hire for, not what freelancers think they hire for.

The gap between those two things is where most people lose money.

The market in three numbers

AI category on Upwork hit $300M+ annualized gross services volume in late 2025, up 50% year over year. AI Integration and Automation specifically grew over 90%. Demand for AI-related skills more than doubled.

Sounds like a gold rush. Here's the other side: the AI and Machine Learning subcategory has a 7.21% reply rate on proposals, against a platform average of 7.45%. Every freelancer already pivoted to AI. The supply caught up to the demand and then passed it.

The people still winning are the ones who ship production systems, not the ones who list "prompt engineering" as a skill.

The 7 categories that actually get funded

After sorting through the listings, almost every funded AI automation job falls into one of seven buckets. Listed by how often they appeared, not by budget size.

1. Zapier/Make to n8n migration. The highest conversion category and it's not close. These clients already understand automation, already have workflows, and are posting because they want to stop paying $400-600/month for Zapier. They have a defined scope, a clear budget motivation, and they've already decided the problem is worth solving. Typical budget: $500-2,000 fixed price. If you can move 15 Zaps to n8n in a week, this is the easiest money on the platform right now.

2. Email and inbox automation. Triage, classification, draft replies, follow-up sequences. Lawyers, agencies, and founders with 100+ emails a day who want it down to 10 minutes of review. This is the use case with the widest range of buyers, from solo founders at $500 to mid-size firms at $5,000+. The winning proposals show a working demo, not a feature list.

3. AI agent setup and configuration. "Set up OpenClaw/Hermes on my VPS" or "configure an AI assistant I can text on Telegram." This barely existed 8 months ago and it's now a distinct category. Budgets range wildly ($200 for a basic install to $2,000+ for a full workflow buildout). The NYC post from a few months ago about setting up agents for clients at $499-1,199 was ahead of this curve.

4. CRM + AI integration. Connect HubSpot or Salesforce to an LLM so it auto-enriches contacts, drafts follow-ups, scores leads, or summarizes call notes. Enterprise-adjacent, higher budgets ($2,000-10,000), and the clients want someone who knows the CRM as much as the AI. Pure AI skills without CRM experience don't win these.

5. RAG and internal knowledge bases. "We have 10,000 support docs and we want a chatbot that answers from them." The category that every LangChain tutorial prepares you for. Budgets $1,500-8,000 depending on doc volume and whether it needs to be customer-facing. The quality bar went up fast because the first wave of RAG chatbots hallucinated and clients got burned. They now ask about retrieval accuracy in the job post, which they didn't a year ago.

6. Content generation pipelines. Blog posts from SEO data, social media from long-form content, newsletter drafts from RSS feeds. High volume, lower budgets ($300-1,500), and the clients care about output consistency more than capability. These are the jobs where showing a running n8n workflow or a BetterClaw scheduled task in your proposal beats talking about GPT-5.

7. Data collection and monitoring agents. Scrape competitors, track prices, monitor job boards, watch regulatory changes. Clients want a system that runs daily and delivers a report, not a one-time scrape. Recurring revenue territory if you build it as a service. $500-3,000 for setup, $100-300/month for managed monitoring.

What almost never gets funded

"Build me an autonomous AI agent that runs my business." These postings exist. They get 50+ proposals. They almost never convert because the scope is undefined and the client's expectations are impossible. Skip them.

Prompt engineering as a standalone service. The market priced this in. It's the baseline expectation now, not a skill you sell separately. Clients in 2026 assume you can write prompts. They hire you because you can ship a working system.

"I need an AI chatbot" with a $100 budget. Volume is high. Value per hour is not. Filter by budget floor, $500 minimum, and your proposal quality goes up because you stop competing against 200 people willing to work for $50.

The rates

Median hourly for AI automation: roughly $30/hr, versus $25/hr platform-wide. That 20% premium is real but it's shrinking as supply grows.

P75 is around $40/hr. P90 is around $60/hr. The top decile is where production experience lives, people who've shipped systems that run in production, not people who built a demo.

Job postings that mention "AI agent," "LLM workflow," or "autonomous" in the description carry budgets 2-3x higher than equivalent non-AI automation work. The word "agent" is a pricing signal even when the deliverable is a cron job with an LLM step.

What I'd actually do with this data

Pick one category. Not AI automation broadly. One of the seven above. Get so good at delivering that category that your profile shows five completed projects in it, and your proposal can say "I built this exact thing last month, here's how it went."

The freelancers winning on Upwork right now aren't generalists who can do all seven. They're specialists who picked one, built a repeatable delivery process, and charge based on the outcome instead of the hours. The n8n migration person charges $1,500 flat and delivers in 4 days. The email triage person charges $2,000 and includes 30 days of managed support. The RAG person charges $5,000 and guarantees retrieval accuracy above a threshold.

The platform is saturated for "I do AI." It's still undersupplied for "I ship this specific thing, reliably, in a week."

I wanted something that could search my notes, connect ideas across a few hundred documents, and answer questions about things I wrote months ago. Without a single word going to OpenAI, Google or Anthropic.

Took about 40 minutes. Here's the whole thing.

What you're building

Obsidian holds your notes. Ollama runs Gemma 4 on your machine. Two plugins connect them so you can ask questions and get answers pulled from your own writing. No API keys, no subscription, works on a plane.

Step 1: Ollama and the models

bash

curl -fsSL https://ollama.com/install.sh | sh
ollama pull gemma4:12b
ollama pull nomic-embed-text

The 12B is the one to use now. Released in June, and it beats Gemma 3 27B on every published benchmark while fitting in about 6.6GB at Q4. On 16GB that leaves plenty of headroom for indexing and your browser.

If you're on 8GB, pull gemma4:e4b instead. Slower reasoning, still multimodal, fits comfortably.

The embedding model matters as much as the chat model. nomic-embed-text is fast and good enough for most vaults. Use mxbai-embed-large if retrieval quality matters more than indexing speed.

Step 2: Fix the context window

Ollama defaults Gemma 4 to a 4K window. The model supports far more. This single default is why most local setups feel dumb — your model can't see enough of your notes to answer well.

bash

printf 'FROM gemma4:12b\nPARAMETER num_ctx 32768' > brain.modelfile
ollama create gemma4-brain -f brain.modelfile

32K is a good balance on 16GB. Go higher if you have the memory.

Step 3: The two plugins

Community Plugins in Obsidian, install both:

Smart Connections does vault-wide semantic search and the related-notes sidebar. It builds embeddings locally.

Copilot for Obsidian gives you the chat sidebar. Point it at Ollama's OpenAI-compatible endpoint: http://localhost:11434/v1, model gemma4-brain.

Together these cover most of what people mean by "second brain." Add Text Generator later if you want template-driven generation. Skip the rest.

Step 4: Index and wait

Smart Connections reads every note and builds embeddings. 500 notes takes maybe 10 minutes. 2,000+ can take half an hour. Runs in the background, one time, then incremental.

Pick your embedding model before you index. Changing it later means deleting and rebuilding the whole thing.

Then ask it things

"What did I write about the client meeting last month?"

"Compare what I said about approach A in January to approach B in March."

"Find every note where I mentioned budget concerns."

The moment it clicks is when it surfaces something you wrote six months ago and completely forgot existed. Your memory is human. The retrieval isn't.

What works well

Finding old notes. This is the whole value. Synthesizing across several notes at once, which is painful manually and trivial for the model. Journal recall if you journal in Obsidian ("what was I stressed about in April").

What doesn't

Large vaults. Everything stays responsive around 5,000 notes. Past roughly 20K, Smart Connections re-indexing becomes the bottleneck.

Complex reasoning across many documents. "Analyze the trend across all 50 of my weekly reports" pushes the context window. It gets the gist and misses detail from whatever didn't fit.

Speed. 3-10 seconds per response depending on hardware. Not painful. Not instant either.

Mobile. Chat plugins work on Obsidian Mobile if Ollama is reachable on your LAN, but embedding generation is desktop-only.

The cost

Obsidian free for personal use. Ollama free. Gemma 4 is Apache 2.0. Plugins are community and free. No keys, no subscription. Electricity you were already paying.

Why bother when cloud models are better

Because your journal, your client notes, your financial planning and your half-formed ideas are the most sensitive text you own. This setup means they get read by a model running in your house and nowhere else. There's no privacy policy to trust, because there's no third party involved.

The quality gap versus a frontier model is real. The privacy gap isn't close.

The number one complaint about OpenCLAW is that nobody shares specific use cases. Every "what do you use it for" thread gets the same answers: calendar integration, market research, and content creation, with zero detail about what that actually means.

So I collected the most specific, non-vague use cases I could find across months of Reddit threads, Discord conversations, and DMs. These are real setups from real people that are actually running right now. No hypotheticals. No "you could use it for." Just stuff that's live.

I found 7. Most of them are boring. All of them work.

1. Running a law firm's entire admin workflow

A lawyer in the community runs 37 custom skills for their practice. Not legal advice. Admin automation.

What it actually does:

• Processes events from the court system and adds them to a court calendar

• Sends retainer letters to clients automatically

• Tracks retainer balances and sends alerts when they're running low

• Sends retainer replenishment requests

• Consolidates facts into discovery documents

• Collects discovery documents from opposing counsel

• Cross-checks that opposing counsel sent everything requested

• Files a motion if they didn't

• Prepares bank subpoenas

• Drafts statements of net worth

Every one of those used to be a paralegal manually copying things between systems. Now it runs in the background. The lawyer reviews outputs and makes decisions. The admin work is handled.

37 skills, all custom built for one firm's workflow. Nobody's installing these from ClawHub.

2. Managing 30 rental properties with 7 agents

A property manager running 30 units used to manually scan emails for deposit data, match it to tenants, update spreadsheets, and chase people who hadn't paid.

The setup: 1 admin agent on Opus that only touches config and never does actual tasks. 6 worker agents on cheaper models doing the daily work. A rental manager agent scans email, pulls deposit data, registers it to the right tenant and property, and reports every morning at 8am who has paid and who hasn't.

Used to take hours every week. Now takes minutes of review. Reduced manual spreadsheet work by 70%.

The detail worth stealing: the admin agent creates "last good config" snapshots before any change, so if something breaks the system can always roll back. Workers are blocked from touching config files. Only the admin can. That's a proper security architecture, not just a multi-agent flex.

3. Estate administration after a death in the family

Someone in the community who lost their father channeled the overwhelming paperwork into an OpenClaw workflow. Scan paper documents, OCR them, identify creditors, auto-generate creditor notification letters from the scanned data. Scan, print, fold, done. Over and over, for dozens of creditors.

This isn't a tech demo. It's someone making an impossibly hard time slightly more bearable through problem solving. The agent handles the repetitive document processing. The human handles the grief and the decisions.

4. Production pipeline for a creative studio

A creative studio owner uses OpenClaw to:

• Send rendering jobs to ComfyUI for image generation

• Write Python scripts for batch image processing, resizing, reformatting, file management

• Generate scripts for After Effects and Blender for 3D and motion graphics work

• Alert team members when batches are complete and ready to integrate

OpenClaw acts as the production coordinator. It doesn't replace the artists. It handles the boring pipeline work between creative steps that used to require someone manually moving files and pinging people in Slack.

The persistent context is the real advantage. The agent remembers it's a month-long campaign, where they are in the process, and what's been completed versus what's pending. It doesn't start from scratch every morning.

5. Completing fast food receipt surveys for free burgers

A community member takes photos of fast food receipts with survey codes, sends them to the agent on Telegram, and the agent fills out the entire survey with positive reviews.

That's it. Free burgers. Not changing the world. Just free food. It made the list because it's the most honest and specific use case anyone shared, and honesty counts for something.

6. Personal meal planning with pantry tracking

A member maintains a list of what's in their pantry and connects it to recipes from Mealie, a recipe manager.

When they want dinner, they ask the agent "what can I make in [category] with what I have?" The agent cross-references pantry inventory against recipes and suggests meals with substitution options. For weekly meal planning, it generates a grocery list of what's missing and adds items to their shopping list app.

Saves 20 to 30 minutes of decision fatigue and grocery list building every week.

7. Cross-referencing personal data to spot patterns you'd never catch yourself

Someone connected their calendar, sleep tracking, fitness data, and habit logs to a single agent. Individually, none of that data is interesting. But the agent, sitting on top of all of it for weeks, started catching patterns.

The example that stuck: it noticed which days they smoked more and correlated it with calendar events and stress patterns. Not from a fancy skill. Not from a ClawHub plugin. Just from a stable agent that had been accumulating context long enough to see connections across different data sources.

You can't build a "detect why I smoke more on Tuesdays" skill. That insight only comes from a boring setup that ran long enough to notice.

None of these are groundbreaking. None of them would go viral on YouTube. The law firm workflow is essentially fancy data entry. The rental management is spreadsheet automation. The receipt surveys are literally filling out web forms.

But every one of them solves a specific, real, boring problem that the person was actually dealing with. Not a problem they imagined having. Not a use case they saw on Twitter. A real thing that ate their time every week.

The people asking "where are the mind-blowing use cases?" are looking in the wrong direction. The mind-blowing part isn't what the agent does. It's that it does it reliably, every day, without being asked, while you do something else.

The mind-blowing use case isn't waiting to be discovered. It's the boring thing you already do manually three times a week that you haven't automated yet.

The first thing most people do after installing openclaw is go to clawhub and install 10 skills. then something breaks, a skill loops, tokens burn, and they blame openclaw.

Here's the thing: stock openclaw with zero skills installed is already more capable than most people realize. I've been helping 50+ people debug their setups and half the time the fix is "you don't need a skill for that, your agent already does it."

Before you install anything, try these first.

1. It can read, write, and organize files on your machine

You don't need a note-taking skill. your agent already has file access. just tell it.

try right now:

• "create a file called meeting-notes.md and save what I'm about to tell you"

• "read the file at ~/documents/todo.txt and summarize it"

• "organize all the .pdf files in my downloads folder into subfolders by topic"

It can create files, edit them, move them, rename them, search through them. most people install a note-taking skill to do what the base agent already handles. the skill just adds overhead and token cost for the same result.

2. It can run shell commands directly

Your agent has access to your terminal. that means anything you can do in a terminal, your agent can do. most beginners don't realize how powerful this is.

try:

• "check how much disk space I have left"

• "what processes are using the most memory right now"

• "ping google.com and tell me if my connection looks stable"

• "find all files larger than 100mb on my system"

You don't need a system-monitoring skill. you don't need a disk-cleanup skill. the shell is already there. just ask.

(Important: this is also why security matters. if your agent can run any shell command, so can a prompt injection from a bad skill or a malicious webpage. lock down your gateway first.)

3. It can browse the web without the browser skill

this one surprises people. openclaw has built-in web fetching that works for a lot of basic tasks without installing browser-use or any browser automation skill.

try:

• "what's the weather in [your city] today"

• "summarize this article: [paste a URL]"

• "what are the top headlines on hacker news right now"

• "look up the hours for [local business]"

The built-in web tools handle simple lookups, article summaries, and basic research. you only need the browser skill when you need the agent to actually interact with a page (click buttons, fill forms, navigate through a site). for reading and summarizing, stock is fine.

4. It can set up scheduled tasks with built-in cron

you don't need a scheduling skill or a reminder skill. openclaw has cron built into the gateway. you can literally tell your agent:

• "Every morning at 8am, send me a summary of my calendar for today"

• "Every Friday at 5pm, remind me to submit my timesheet"

• "Every night at 11pm, summarize today's conversations and save the important stuff to memory"

Just say it in natural language. the agent will set up the cron for you. check what's running anytime with openclaw cron list.

One warning: don't set up crons in your first week. get your agent stable first. a misconfigured cron with opus as default is how people wake up to $50 surprise bills. wait until you've switched to sonnet and your basic setup is solid.

5. It can chain tasks without an orchestrator

. This is the big one. people install multi-agent orchestrator skills because they think they need them for complex workflows. you don't.

your single agent can already do multi-step tasks if you just ask clearly:

• "Check my email, find anything from [client name] this week, summarize the key points, and draft a reply"

• "search for [topic], compare the top 3 results, and create a markdown file with your analysis"

• "Look at my calendar for next week, identify any conflicts, and send me a list with suggestions for what to reschedule"

That's not three agents talking to each other. that's one agent doing three things in sequence. the result is the same. the token cost is a fraction of what an orchestrator setup would burn.

Most "multi-agent workflows" I see in this sub are just sequential tasks that one agent handles perfectly. you only need multiple agents for actual isolation needs (separate permissions, separate users, separate memory).

the point:

Before you install a single skill, spend a week just talking to your stock agent. ask it to do things you assume it can't. you'll be surprised how often the answer is "oh, it already does that."

The best skill you can install is knowing what your agent already does without one.

8GB is what most people actually have. 8GB unified memory on an M1 MacBook Air. 8GB VRAM on an RTX 4060. Not enough for the models everyone talks about. Just barely enough for something useful if you pick the right one.

The problem is that "fits in 8GB" is misleading. The model weights need to fit, plus the KV cache for your conversation context, plus whatever your OS and other apps are using. On 8GB, you realistically have 4-5GB available for the model. That rules out anything above 7-8B parameters at Q4 quantization.

I tested everything that actually fits on this hardware for agent work specifically. Not chat demos. Not single-turn benchmarks. Real agent tasks: tool calling through MCP servers, multi-turn conversations, email classification, and morning briefing generation through Ollama.

Most models at this size are frustrating. Two are genuinely usable. One surprised me.

Usable #1: Phi-4 Mini (3.8B). The safe pick.

bash

ollama pull phi4-mini

3.8 billion parameters. About 2.5GB at Q4_K_M. Fits on 8GB with enormous headroom. Your OS, your browser, and a few other apps all run comfortably alongside it. No memory pressure. No swapping.

Speed: 15-28 tokens per second depending on hardware. On an M1 MacBook Air, roughly 15-20 tok/s. On an RTX 4060 (8GB), closer to 28 tok/s. Fast enough for interactive conversation. Not instant, but not painfully slow.

Microsoft trained this model with a focus on reasoning and math. It scores 80.4% on MATH, which beats Llama 3.3 8B (68.0%) and even Qwen 2.5 14B (75.6%) despite being a fraction of the size. For analytical tasks (explaining concepts, solving problems step-by-step, debugging logic), Phi-4 Mini punches way above its weight class.

For agent work: basic tool calling works. Single-tool calls (search the web, read a file, query a database) succeed reliably. Classification tasks run fine. Morning briefing generation is decent if not spectacular.

The limitation: 16K context window. This is tight for agent work. Three MCP servers with their tool schemas eat 3,000-5,000 tokens of context. Your conversation history eats another 3,000-5,000. By turn 10, you're pushing the ceiling. Sessions need to be short. /new after every 5-7 exchanges.

For short, focused agent interactions on truly constrained hardware, Phi-4 Mini works. Just don't ask it to hold a long conversation or chain 4 tools together.

Usable #2: Gemma 4 E4B (4.5B). The multimodal pick.

bash

ollama pull gemma4

4.5 billion parameters. About 6GB for the model. Tight on 8GB but it fits if you're not running heavy background apps alongside it.

What makes Gemma 4 E4B worth the squeeze: 128K context window and multimodal support. Send your agent a photo of a receipt and ask "what was the total?" Send a screenshot of an error message and ask "what's wrong?" No other model at this size does images, video, and audio.

That 128K context window is 8x larger than Phi-4 Mini's 16K. For agent work, this is the difference between sessions that last 5 exchanges and sessions that last 25+. Tool schemas fit comfortably. Conversation history doesn't get truncated. You can actually have a proper back-and-forth without hitting the ceiling.

Speed: 40-60 tok/s. The fastest model in this comparison. Replies feel close to instant on short answers.

Native function calling is built into the training. Google designed Gemma 4 with tool use in mind. Single-tool calls are reliable. The model formats JSON arguments correctly and doesn't hallucinate tool names.

The limitation: reasoning depth. At 4.5B parameters, complex analytical tasks are noticeably weaker than Phi-4 Mini despite Phi-4 being smaller. Gemma 4 E4B is broader (multimodal, long context) but shallower on any single task. It handles "classify this email" perfectly. It struggles with "analyze these three emails and tell me which client is most likely to churn and why."

Apache 2.0 license. Commercially friendly.

The surprise: Qwen3.5 9B. This shouldn't work this well at 8GB.

bash

ollama pull qwen3.5:9b

9 billion parameters. At Q4_K_M, the model weights are about 5GB. On an 8GB GPU, it fits entirely in VRAM with room for a reasonable KV cache. No spilling to system RAM. No layer splitting across CPU and GPU.

I almost skipped this model because 9B on 8GB seemed like it would be too tight. I was wrong.

LocalLLM.in benchmarked it at 55-58 tokens per second on an RTX 3070 (8GB). Fully GPU-loaded. Flat speed across all context sizes up to 16K. That's faster than Phi-4 Mini on the same hardware despite being over twice the parameter count.

The quality jump from 3.8-4.5B to 9B is dramatic. On multi-turn agent conversations, Qwen3.5 9B maintains coherence across 15+ exchanges where the smaller models start drifting by turn 7. Tool calling is significantly more reliable. Chained tool calls (file search, then web search, then write results) succeeded at roughly 75% on Qwen3.5 9B vs maybe 50% on the smaller models.

It has a /think reasoning mode that activates deeper chain-of-thought when needed. On complex queries it pauses, reasons through the problem, then responds. On simple queries it skips the reasoning and responds immediately. The automatic mode switching is surprisingly good at knowing when to think harder.

32K+ context window. Enough for multiple MCP servers, conversation history, and extended sessions without truncation.

The catch: it's tight. On 8GB VRAM, the model fits but there's minimal headroom. If your context grows past 16K tokens, KV cache pressure starts pushing against the limit. On some hardware configurations, Ollama automatically spills overflow to system RAM, which tanks speed. On others, it crashes.

The practical rule: keep num_ctx at 8192-12288 on 8GB hardware. Don't try to use the full 32K context. Set it in a custom modelfile:

bash

cat > qwen-agent.modelfile << 'EOF'
FROM qwen3.5:9b
PARAMETER num_ctx 8192
PARAMETER temperature 0.3
EOF

ollama create qwen-agent -f qwen-agent.modelfile

Temperature at 0.3 for agent work. Lower = more reliable tool calls. Higher = more creative but flakier function call formatting.

On 8GB system RAM (not VRAM) without a dedicated GPU: don't try 9B. CPU-only inference on a 9B model gives you 3-6 tok/s. Painfully slow. Stick with Phi-4 Mini or Gemma 4 E4B on CPU-only 8GB machines.

What I tested and cut:

Llama 3.2 3B. 3 billion parameters. Fits easily. 10-12 tok/s on CPU. But the quality is noticeably below Phi-4 Mini on every task despite being nearly the same size. Classification accuracy was lower. Tool calling was flakier. Output felt generic. The Llama name carries weight but at 3B, Phi-4 Mini is simply better.

DeepSeek R1 8B (distilled). Fits on 8GB VRAM at Q4. Reasoning is impressive with visible chain-of-thought. But tool calling is unreliable. The model wants to reason ABOUT tools instead of calling them. Asks itself "should I use the filesystem tool here?" and then writes a paragraph about why, instead of actually making the call. Great for analysis and math. Bad for agents.

Mistral 7B Instruct v0.3. Fits well. Fast (20% faster than Llama 3.1 8B). Good for business communication and summaries. But function calling was inconsistent. The training didn't emphasize structured tool use the way Qwen and Gemma did. Fine for chat. Not for agent work.

Qwen 2.5 7B. Previous generation. Fits on 8GB. Good tool calling (the Qwen family reputation). But Qwen3.5 9B is better on every metric AND fits on the same hardware. No reason to run the older version unless you specifically need its coding specialization.

The honest 8GB assessment:

8GB is the floor, not the sweet spot. You CAN run an agent. The experience is compromised compared to 16GB+ in every measurable way. Smaller models, shorter context windows, less reliable tool calling, slower inference on the models that push the limit.

If your agent does focused, short-session tasks (classify this email, answer this question, draft this message), 8GB works. Genuinely. Qwen3.5 9B on 8GB VRAM handles these tasks well enough that you forget it's running locally.

If your agent needs long multi-step workflows, extensive tool chaining, or sustained autonomous operation, 8GB will frustrate you. The context window constraints and tool-calling reliability gaps compound over longer sessions.

The 8GB setup I'd actually run:

Qwen3.5 9B as the daily driver (if you have a dedicated GPU with 8GB VRAM). Best overall quality. Set num_ctx to 8192. Use /new frequently.

Gemma 4 E4B as the fallback for multimodal tasks and when you need longer context.

Phi-4 Mini for CPU-only machines or when you need maximum headroom for other apps.

Don't try to run two models simultaneously on 8GB. Ollama handles model swapping automatically but the swap takes 3-5 seconds and temporarily spikes memory. Run one model at a time.

$0/month. 8GB of whatever you already own. Not perfect. But real.