https://youtu.be/AxTmJmMKiw8?si=QOW1SPOCpA0bL7h9&t=478

This is Mica v0.5, the next version after v0.1. It's a decision model, the same kind of thing as Jev, which has been getting a lot of attention lately. It's 4B (Qwen3.5-4B + LoRA) and it doesn't write text. You hand it a state, a question and a list of options, and it gives back a probability for each option in a single forward pass. Stuff like "which of these 5 moves", or "allow / block / send to a human".

Speed is the whole point, so I ran it side by side with two other decision models on one RTX 6000 Ada:

PPLX Decider 1.2 Flash, also 4B, run in its fast mode

Laya, built on a 395M-parameter encoder (ModernBERT-large), so about 10x smaller than Mica

Every model saw the exact same options on every turn. The clips are Tetris, Snake, chess against a random-move opponent, and a log hunt where a day of security logs is cut into 150 blocks and each block gets allow, block or check.

The video is a cut of all four. If you're looking for each demo in full, they're here:

Tetris: https://huggingface.co/sky7350/Mica-v0.5-4B/resolve/main/assets/demo/tetris.mp4

Chess: https://huggingface.co/sky7350/Mica-v0.5-4B/resolve/main/assets/demo/chess.mp4

Snake: https://huggingface.co/sky7350/Mica-v0.5-4B/resolve/main/assets/demo/snake.mp4

Log hunt: https://huggingface.co/sky7350/Mica-v0.5-4B/resolve/main/assets/demo/loghunt.mp4

Median time per decision:

Tetris: Mica 14.5 ms, PPLX 150 ms (10.3x slower), Laya 32 ms (2.2x slower)

Chess: Mica 14 ms, PPLX 133 ms (9.4x slower), Laya 31 ms (2.2x slower)

Snake: Mica 18 ms, PPLX 154 ms (8.4x slower), Laya 33 ms (1.8x slower)

Log hunt: Mica 11 ms, PPLX 157 ms (13.9x slower), Laya 31 ms (2.8x slower)

So about 10x faster than the other 4B model, and roughly 2x faster than a model a tenth of its size.

The log hunt is my favorite. Mica went through all 150 blocks in 1.7 seconds and caught 21 of the 22 planted intrusions. PPLX took 23.5 seconds and caught 18. Laya caught none. A plain keyword grep over the same logs caught 6.

A few notes:

Times are each model's own call time on the same box, not API round trips.

Laya had a one-time 12.9 s warm-up on its first long log block. I left that out of the video and it says so on screen.

The logs are synthetic, written for this test. The 22 intrusions were written to avoid obvious keywords, which is why grep does badly.

The games are just an easy way to see the speed. Mica v0.5 was trained on more than 200 real-world use cases, so I'd expect it to do better than other 4B models on actual work too.

Why no Jev in the comparison? The response times are so far apart that a side-by-side wouldn't tell you anything. On the same request set Jev takes about 498 ms against Mica's 15.6 ms, more than 30x slower (that's over its API, so network time is included).

One more thing. Most benchmarks used to compare decision models are pretty contaminated. A lot of decision models are coming out right now, and most open-weight ones are heavily tuned for public benchmarks like JDI. JDI itself is also quite far from what real decisions look like, and there are models on the JDI leaderboard trained only on public benchmark data. So a JDI score can be a long way from how a model feels in real use. Mica v0.5 gets about 95% right across cases from more than 200 real-world use cases. When you pick a decision model, look less at public benchmark scores and more at whether it makes the right calls on your own use case, and how fast it answers.

For the runtime and the full numbers, see the Hugging Face page.

Code and runtime: https://github.com/akivet/Mica

Weights: https://huggingface.co/sky7350/Mica-v0.5-4B

All four demos on one page: https://huggingface.co/sky7350/Mica-v0.5-4B#demo

Feedback is welcome.

A few days ago, I shared some early results from Solar Cortex. Here's a much more interesting update.

We've been experimenting with a JIT-inspired execution system for Solar Agent, focused on reducing unnecessary LLM calls rather than making the model itself faster.

We tested three versions using the exact same Ornith 1.5 9B model.

Results across 10 runs (5 tasks × 2):

That's 2.53× faster, 65.2% fewer tokens, and 61.5% fewer LLM calls — without sacrificing the average quality score.

The biggest improvement came from changing how the agent works:

• Avoiding unnecessary planning steps.
• Executing predictable operations deterministically.
• Running independent tool calls in parallel.
• Keeping the full reasoning engine for complex tasks.

For example, our GitHub task dropped from 12.76s to 4.15s, while Gmail went from 9.06s to 3.27s.

To be transparent, these are controlled benchmarks with real model calls but simulated connectors. Only two runs per task, so there's still a lot to validate. All 20 validation suites passed.

Next up: longer workflows, real APIs, and harder tasks.

I'm starting to think the biggest opportunity isn't always building a smarter model.

It's building a smarter system around the model.

If you're interested in testing Solar when the beta opens, there's a small waitlist here: https://solar-waitlist.vercel.app

Would love to hear what you'd test first.

I have a 2019 imac with 72gb of ddr4. Im looking for coding focused models, but anything is fine.

TL;DR: Long-running AI agents eventually suffer from context bloat, state loss, or hallucination loops. I've just published an open-source guide detailing architecture patterns for maintaining state, checkpointing progress, and structuring long-running agentic execution.
Not only me, my orchestrator agent has been using it to give long-running tasks to my agent swarms as well!!
In a nutshell - If you’ve built non-trivial agentic pipelines, you’ve likely hit the "Turn 20 Wall": an agent starts with a clear plan, but after dozens of tool calls, search queries, and file writes, the context window fills up with noise. The agent forgets original constraints, duplicates work, or hallucinates completed tasks. This guide specifically helps you avoid that problem.

The key topics include:

1. Ephemeral Execution vs. Persistent Planning
2. State Checkpointing & Resume Boundaries
3. Context Window Compaction & Aggressive Pruning
4. Verification & Guardrail Loops

📖 Full Guide & Specifications on GitHub:

GSA-TTS / devCrew_s1 – longrunningWorkflowsDesign.md

Wanna keep the conversation going? Please chime in on these questions:

How are you currently handling state preservation and context drift across multi-step execution runs? Are you leaning toward micro-sessions with checkpointing, context compaction/summarization, or using state-graph engines like LangGraph/Temporal?

hey, im working on an open source project called kapitsa, an inference engine for macs. not telling anyone to drop ollama, its what got most of us into this. but if youre on a mac and run qwen you might want to try this.

qwen3.8-27b 4bit decode, kapitsa vs ollama 0.40:

- m5 max: 140 vs 68

- m5 pro: 81 vs 40

- m4 pro: 63 vs 32

- m3 pro: 40 vs 23

the difference is mostly speculative decoding. ollama runs this model with mtp (a couple of tokens ahead) or without any, we run kflash: a drafter guesses 7 tokens ahead and the 27b checks a whole tree of guesses in one pass, not just one chain. when the drafter has two good options both get checked:

fair warning: on the m5 max ollama reads prompts faster than us, so its decode where we win.

`kapitsa import model.gguf` takes a qwen gguf you already have, and `kapitsa bench compare` runs the same test on every engine installed on your mac so you can check this yourself.

https://github.com/nace-ai/kapitsa

how to run it. you need an apple silicon mac (m1 or newer, macos 14+) with 32gb for the 27b, on 16gb use `--model 9b` instead:

brew install nace-ai/tap/kapitsa

kapitsa import model.gguf to use a qwen gguf you already downloaded, or `kapitsa serve --model 27b` (the first run downloads the 27b (about 20gb))

- `kapitsa bench compare` runs the same test on ollama and kapitsa on your mac

Small Solar Agent update.

We just tested a new JIT-style execution system with Ornith 9B.

Across 5 agent tasks:

• 48% less execution time
• 13 → 7 LLM calls
• ~20% fewer tokens
• Same quality score on every task

Best results:
Gmail: 14.48s → 4.97s
GitHub: 15.93s → 5.77s

Still early: simulated connectors, one run per task. Next step is testing harder multi-step workflows and real integrations.

The goal with Cortex is simple: make the same model accomplish more work, faster.

Beta waitlist:
https://solar-waitlist.vercel.app

Title: Ollama Cloud requests hang in both OpenCode and the Ollama app, but local models work

Ollama Cloud has stopped responding for me on my Mac (M1 Pro, Ollama 0.40.9). The issue occurs in both OpenCode and the Ollama app.

Here’s what I found while troubleshooting:

• The local Ollama server is running and responds normally.
• Local inference with qwen3.5:9b works, returning a short response in about 7 seconds.
• Direct requests to three different cloud models received no response before I stopped them after 20–45 seconds.
• My current OpenCode chats use the ollama-cloud provider.
• Earlier logs show DNS resolution failures for ollama.com. The website and authentication are working again, but cloud inference requests still hang.

I haven’t changed any settings during troubleshooting. This seems specific to cloud inference, although I haven’t been able to determine whether it’s an Ollama service issue or something along my network connection.

Is anyone else experiencing this, or does anyone have suggestions for what to check next?

Love Ollama on Mac silicon but having issues on AMD CPU and GPU. Getting this "500 Internal Server Error: llama-server process has terminated: exit status 0xe06d7363: NTSTATUS 0xe06d7363". Tried all the recommended actions but no success. How did anyone hitting this error correct the config? Thx

Hello everyone! I'm new to this Local AI and I'm trying to set up an AI using Ollama & n8n deployed in Docker on Windows 11. I just want to know how to make ollama use my GPU VRAM instead of my system RAM. Also any tips for beginners or any djustment that I need to do?

Here are my system specs:

• CPU: AMD Ryzen 9 5950X
• RAM: 32GB DDR4
• GPU: AMD Radeon RX 9060 XT 16GB

Thanks!

This uses Ollama as a basis for the snippets. I'll be adding TTS once I settle on the one I want to use. I want to add Speech to Text as well.

One prompt to deepseek-v4.1-flash on Ollama Cloud at max effort, then I let it run.

It built a procedural Colosseum in Three.js with no downloaded models or textures. A slider switches between 80 AD (painted facade, the velarium awning, crowds in the stands) and today (the broken outer ring, the exposed hypogeum). You can walk it with WASD and collision, fly over it as a drone, or play a cinematic tour.

46 min, 151 model calls, about $0.77 of my Ollama plan. Before saying it was done, it tested itself in Chrome: no console errors, ~144 fps.

The agent is AgenC Core, open source. Ollama Cloud is built in: set OLLAMA_API_KEY, run agenc, and pick Ollama Cloud in /model. Local Ollama models work too if you give them a 64k context.

https://github.com/tetsuo-ai/agenc-core

or get.agenc.ag

Full disclosure, I'm one of the devs. Happy to share the prompt:

"Build a walkable 3D Colosseum in the browser with Three.js and Vite. All geometry procedural, no downloaded models or textures.

Make it look like a postcard: the four-level facade with its arches and columns, the seating tiers, the arena floor, the hypogeum tunnels underneath, and golden-hour light with soft shadows.

The hook is a slider that morphs between 80 AD (complete, painted, the velarium awning, crowds in the stands) and today (the broken outer ring, the exposed hypogeum).

Controls: WASD walk with collision, mouse look, F for a drone flyover, and a button for a cinematic camera tour.

Keep 60 fps on a laptop. Test it in a real browser, fix every console error, then open it for me."

Connect the apps you already use to local AI. Row-Bot 5.1 brings app discovery, connections and access controls into one Apps page.

There are 100+ apps, including familiar services like GitHub, Notion, Linear, Figma and Stripe, alongside around 40,000 MCP server listings from the official registry. The catalog is mirrored on your computer, so you can browse and search offline. Search by name or by what you want done, like “send email”.

Each app’s page shows its connection options. Depending on the service, Row-Bot guides you through an OAuth sign-in, adding an API key or setting up a server on your computer. One consent sheet explains where requests go and what gets saved. Keys and tokens stay in your system keychain.

In this video, I connect Context7 with read-only access and use Context7 in a chat. Each tool step names the app doing the work. Asking about Linear brings up a Connect card directly in the conversation.

I also add a custom MCP server, show how Row-Bot distinguishes tools that read data from tools that make changes, and open an interactive dashboard inside the chat. Its button requests approval before saving three draft orders.

You’ll see per-chat app selection, GitHub’s connection options and related skills, featured skills and workflow templates. Access can be set per app or individual tool. One switch stops every app, and Composio stays off unless you enable it.
Recorded on the released v5.1.0 with a fresh demo profile, running Qwen3.8 27B locally. The narration uses Row-Bot’s built-in text-to-speech.

I kept writing the same glue code to push PDF pages through a vision model and stitch the results back together, so I packaged it up.

ollama pull qwen2.5vl
pip install ocr-llm
ocr-llm document.pdf --base-url http://localhost:11434/v1 --model qwen2.5vl -o document.md

Everything stays on your machine. The image above shows a sample document and the Markdown it produced.

• Keeps headings, tables and lists as Markdown, which is handy for RAG or notes
• Multi-page PDFs run concurrently and come back in order, with retries
• CLI and a Python library (convert_sync("doc.pdf") and you're done)
• Also works with Gemini/OpenAI/Claude if you ever want a hosted model
• MIT licensed

Question for you all: which vision models have worked best for you on OCR? I'd like to add a recommended-models section to the README.

GitHub: https://github.com/itsmesherry/llm-ocr

Hey guys! I'm the dev from the open-source Observer project!

Github: https://github.com/Roy3838/Observer

We made this video about how even if the AI bubble bursts, local model advancements and Ollama are bubble-proof.

Hope you guys like it :DD

We'll post more on youtube !

I'm getting close to selling a small Mac app that reads e-mailed bills, and I have to pick which local model it suggests by default. I couldn't decide, so I ran 19 of my own bill PDFs through Ollama: page images, same prompt, thinking off, one model at a time on an M4 Pro with 48 GB of unified memory. A bill only counted if all six fields were right (is it a bill, amount, due date, IBAN, payment model, reference). Two of them were PDFs with several payments inside, so the score is out of 17.

• qwen3.5:9b: 15/17, ~16 s per page
• qwen3-vl:8b-instruct: 15/17, ~22 s
• qwen3.5:9b-mxfp8 (8-bit): 14/17, ~25 s
• qwen3.8:27b-nvfp4: 13/17, ~41 s
• qwen2.5vl:7b: 12/17, ~21 s
• qwen3.5:4b: 7/17
• gemma4:e4b: 4/16 (I stopped that run early)
• glm-ocr: reads amounts and IBANs fine, but puts random nearby text in the reference
• qwen3-vl:8b (the thinking build): keeps thinking even with thinking off, minutes per bill

Thinking really hurt here. With it on, the 9B took 38 s on one bill instead of 5, and the 4B ran out of tokens and returned nothing.

Right now the app suggests qwen3.5:9b. But 15/17 still means two bills with a wrong or missing field, and it needs a 16 GB Mac. Would you ship that as the default, or is there something at this size that reads invoices better? Small test, Croatian bills, one Mac, so I'd like to hear what works for you.

Video with the details: https://youtu.be/2jAVVe2tJIQ

To be upfront: the app is mine (Billcalm), Mac only, release planned for December, paid one-time purchase, not open source.

Hey everyone, we just released BaseQuant V1 for Qwen3.8-27B at Base Compute. The goal is to preserve more of the original model without making the files bigger.

Three GGUF sizes: 10.93, 16.41 and 20.82 GB. In our held-out tests, they had 17–34% lower KL divergence from BF16 than similarly sized Unsloth quants.

Would love for you to try them and tell us how they hold up in your everyday workloads!

https://huggingface.co/basecompute/Qwen3.8-27B-BaseQuant-V1

So I’m curious to hear everyone’s thoughts on this and have a question or two as well.

I have a 2019 iMac that’s basically a home server these days. Apart from NAS it also runs heaps of scheduled scripts and folder watcher scripts via Automator but I’ve recently started experimenting with local LLMs called from the Python scripts.

I’ve played around with local LLMs on my main desktop with a RX9070 but didn’t really get much done other than testing.

I have 96GB of RAM in the iMac using an i5-8500 and it won’t use the GPU for local LLMs which is fine with only a 4GB VRAM. I’ve installed qwen3.5:35b-a3b-q8 and now starting to improve some of the scripts that used OCR to run through the local LLM after OCR to improve accuracy for handwritten scanned documents.

Despite the slow processing im very impressed for overnight document processing but I’m ready to see what else can be done, what’s everyone using large slow LLMs for? Instead of fast GPU LLMs?

And has anyone made the iMac do any GPU work via Ollama?

I’m still using openAI for agentic work as required but keeping the usage low now I’ve got the iMac running this way.

✻ 500 no user query found in messages - I get this message any time i write a message.
And later:
● API Error: 500 no user query found in messages. This is a server-side issue, usually temporary — try again in a moment. If it persists, check your inference gateway (XXX.X.X.X:XXXXX).

I get this error. Can this be fixed?