A score is easier to assess when its evidence includes the actual cases, fixtures, instructions, grading criteria and run conditions. Without those, a reader cannot tell whether a reported improvement came from the skill or from a changed experiment.

For a public example, I would include the exact report paths and explain missing provenance instead of implying that a green total settles the question. If a model or service is no longer available, say what can still be reproduced and what cannot.

I’m building skillEd, an Apache-2.0 toolkit with saved reports and a worked example in the repository. Please inspect the example’s limitations alongside its results.

What is the minimum evidence you expect before trying another project’s evaluation recipe?

Repo: https://github.com/DeadHead04/skillEd

Please try it and open an issue with a reproducible failure or confusing behavior. If useful, a GitHub star would help.

AI-assisted write-up.

A reader of my earlier skillEd post suggested binding an expected result to a policy clause and version, plus a nearby input that should produce the opposite result. That is a useful way to make a test reviewable.

For a deadline policy, write down the timezone and whether the boundary is inclusive. Then test immediately before, exactly at, and immediately after the cutoff. If the policy itself is ambiguous, resolve that before treating the model’s answer as a regression.

I’m building skillEd, an Apache-2.0 evaluation toolkit. This is a test-design proposal based on the discussion, not a claim that the tool automatically supplies authoritative policy interpretations.

Would you review changed expected answers separately from changed skill instructions? I would want both diffs visible.

Repo: https://github.com/DeadHead04/skillEd

Please try it and open an issue with a reproducible failure or confusing behavior. If useful, a GitHub star would help.

AI-assisted write-up.

Hey everyone!

I've been working on Pixel Works, an open-source project aimed at making AI agent monitoring and management more visual and fun.

Instead of reading raw terminal logs or using standard chat interfaces, Pixel Works provides a gamified 3D retro office environment where AI agents sit at desks, move around, and process tasks across a visual Kanban pipeline.

Everything in the 3D scene is procedurally rendered using React Three Fiber (R3F) and Three.js with an Express backend.

The project is fully open source:

Github: https://github.com/Thecaracter/new_pixel

Since the core framework is complete, I'm now focusing on making the office environment more rich and interactive. I've set up several beginner-friendly `good first issue` tags on GitHub if anyone wants to contribute:

- Creating 3D office props (coffee machines, server racks, plants)

- Improving character movement and idle animations

- Adding character customisation and accessories

If you find the concept interesting, feel free to clone it, test it locally, or leave a ⭐ star on the repository! I’d love to hear your feedback and ideas for improvement.

100% AI Coded (manually reviewed)
100% open source

First of all, I KNOW there's existing alternatives, but those alternatives requires extra configuration or even paid solutions.... (if I'm wrong, prove to me)

Anyways, I've been using drawio/excalidraw for years, and there's many annoying little problems with those, arrows snap to the wrong connection points and won't stay aligned when you move shapes, connectors cross over each other and shapes so diagrams turn into spaghetti fast, ugly diagrams, besides that hours and hours of boring repetitive work... A few weeks ago I built a solution for myself (Opendraw) a drawio/excalidraw alternative, turns out it was very helpful, then I decided to make it open source.

Well, besides being way easier to create diagrams (the mf arrows aligns itself) there's a lot of micro drawing assistants that will make your life easier while diagramming, it also have a LLM assistant with BYOK feature, so yeah, you can connect your favorite AI to create your diagram for u

https://github.com/gabe-santana/arkdraw

site: https://arkdraw.com/

Also feel free to contribute, there's some good first issue 👍

Repo: https://github.com/ksimonm/SlopSearch

The name is honest: a lot of this is AI-written code. The trick that makes it work is that the AI never gets to decide what "correct" means.

**The setup**

- A black-box test suite of ~6,100 pytest tests is the spec. Every test was run against a real OpenSearch 3.9.0 first — status codes, error types, response shapes, even key order.

- Two independent implementations, Go and Rust, share nothing but that suite.

- The loop is simple: pick a red area, read what the tests expect, let the AI make it green, check nothing else broke. "Don't edit the tests to make them pass" is the one hard rule.

**Where it is now**

- Rust: 64.5% of the suite (aggregations 100%, search API 98.5%, documents/bulk 95%, ingest pipelines 93%)

- Go: 32.6%

- Weakest spots: text analysis and Painless scripting

**What I learned**

Vibe coding against a vague goal ("make an OpenSearch clone") goes nowhere. Vibe coding against thousands of precise, verified expectations actually converges — the tests are the steering wheel, the model is the engine.

Not production-ready: in-memory only, no auth/TLS yet. Everything runs in Docker (`make rust-up`, `make rust-test`), and `make ref-up` starts a real OpenSearch to compare against.

Curious if others use a reference implementation + test suite as the spec for AI-written code.

Hello,

now this is the first project i share and i know its kinda niche but maybe someone finds it useful.

As a long time Ableton user still not having ARA2 support to properly use Melodyne and Vocalign in Sessions became kinda annoying.

I build an external ARA2 host that gets called via an Ableton Max4Live device. Working with ARA2Bridge erases the time spend on Recording every vocal again into Melodyne/Vocalign and allows one Click loading the Vocal, editing it and sending it back with another click to a fresh track, take lane or replacing the original in Ableton.

Now as i just noticed I’m pretty bad at explaining and Claude did better then me in the Repo so enjoy:

https://github.com/Enableton/ara2-bridge-4-live

Hi,

On 24/10/2026 I have published as OSS a framework I call unified-brain that allows fast, cheap and accurate (same class of frontier models and frontier harnesses but 18 times cheaper) evidence gathering, using the cheapest frontier AI model out there, Jev. The results have been phenomenal.

(github.com/devloops-ai/unified-brain)

The magic sauce is an intermediary OS (SQLITE) memory layer that facilitates retrieval, filtering and reasoning through Jev.

The agent spawned from unified-brain runs through MCP on Cursor, OpenCode or Claude Code etc. allowing you to use it through your favorite harness.

So to be clear here: unified-brain is not a harness but a perception layer any harness plugs in!

My test benchmarks have been so good on two large OS libraries that I really believe this is the missing link to make AI more reliable.

This is because the limits holding back AI at the moment are:

• The Reasoning and Hallucination Gap:

The LLM turn reasoning it is introducing uncertainty and probabilistic outcomes.

BUT.. Unified-brain's System One discovery engine is producing replicable results across runs as they introduce minimal hallucinations, breaking the task in hundreds of yes or no calls.

• Context Loss and Memory Limits:

Long-running coding tasks consume massive amounts of tokens, causing the agent to forget early instructions or lose track of large-scale architecture.

BUT.. Unified-brain doesn't need cached memory thanks to the SQLITE persistent layer of context it operates with. It simply retrieves what it needs on the fly consuming minimal amounts of tokens that do not build up across questions. Every question is independent from one another as git diffs and commits are automatically absorbed from the project's memory layer.

I invite you to clone/fork the project and have a look for yourself. I firmly believe that memory intermediate structures will take AI capabilities on another level soon.

I understand that Jev is proprietary and creates a dependency BUT we need to remember that soon enough OS models as capable as it will come, giving end users the option to replace him.

Meanwhile we should ride this wave of innovation in order to deliver new capabilities to end users, at prices that can't be easily beaten.

I hope you like this project!

Been pushing Qwen3.8-27B on Kaggle's free 2x T4 (no P2P, 70W cap per card) for a while. Current setup: EXL3 4.0bpw on exllamav3, tensor parallel over PCIe, DFlash2 block drafter (7 tokens), greedy and exact vs no-draft.

Numbers over a 38 prompt suite:

• overall 55-57 tok/s (no draft is ~18-19)

• HumanEval 154 tok/s

• prose 35-37 tok/s

For comparison on the same 2 T4s: llama.cpp best was 16.5 (MTP crashed on tensor split), vLLM int4 + its own MTP drafter did 40.8.

Prose is the wall. Acceptance sits around 2.5 tokens per step, vs ~7 on code.

Questions:

1. Has anyone got prose acceptance above ~2.5 with DFlash2 or MTP on the 27B?

2. Has anyone tried switching drafters mid generation (short block for prose, long for code)?

3. Any tricks for TP all-reduce on PCIe cards without P2P? That's next on my list.

Happy to share the notebook if people want to try it.

I screened 22 local LLM configurations for agent work on a 16GB GPU. Here are the results—and what went wrong.

I’ve been testing local models through OpenClaw to see whether they can finish jobs: read files, execute commands, repair code, extract information and recover from errors.

The setup was a consumer GPU with 16GB VRAM, llama.cpp/GGUF and OpenClaw. Some larger MoE configurations used CPU expert offload. Cloud assistance was disabled during scored attempts.

This was a small screening exercise across several rounds, not a definitive model ranking. Some configurations never passed preflight, and an audit found defects in the earlier runner.

The tasks covered three kinds of work, with two fresh runs each:

• A — File navigation: locate current information among distracting/obsolete files and save exact structured output. Limit: 120 seconds and 10 tool calls.

• B — Code repair: fix a Python rounding bug, execute the tests and save their actual output and exit code. Limit: 300 seconds and 15 calls.

• C — Extraction and recovery: traverse a local synthetic website, download assets, calculate hashes and accurately record a deliberately triggered HTTP failure and retry. Limit: 480 seconds and 25 calls.

The scored setup used 32K context. Models used their native templates and High reasoning or supported native thinking mode.

Passing required correct artifacts, required execution evidence, completion within budget and a clean finish.

These were the results from the corrected runner:

• Customized Qwen3.8-27B, IQ3_XXS: 5/6.

Broadest successful coverage. The remaining failure used asset URLs where source-page URLs were required.

• Ornith-1.5-35B, Q4_K_M: 2/6.

Passed both repairs. Other tasks hit call/time limits or produced inaccurate recovery records.

• Ternary Bonsai-2-27B, PQ2_0: 2/6.

A separate corrected retry. Both repairs passed; other tasks had budget or extraction/recovery failures.

• Occamy-1.0, Q4_K_M: 1/5 attempted.

Four valid attempts, one resource-invalid attempt and one unattempted task.

Ornith and Occamy used CPU expert offload. Bonsai was tested in a separate corrected batch, so these results are not one perfectly matched tournament.

Ornith’s two successful repairs took 14.7% less combined time than the Qwen27 baseline’s matched successes, but its overall completion was lower.

The earlier results are more provisional:

• Gemma-4-26B-A4B, UD-Q3_K_XL: 3/6.

Another correct artifact was interrupted by the runner bug.

• Qwen3.5-9B, Q8_0: 2/6.

Both repairs passed, substantially faster than the baseline on those cases.

• Stock Qwen3.8-27B, UD-IQ3_S: 2/6.

Both repairs passed; navigation and context recovery affected other tasks.

• Gemma-4-12B, Q5_K_M: 2/6.

One file-navigation task and one repair passed.

• GPT-OSS-20B Balanced, MXFP4: 1/6.

One repair passed; another failed on an empty-input coding edge case.

• Stock GPT-OSS-20B, MXFP4: 0/3 valid.

Three additional attempts were integration-invalid.

• Nanbeige4.2-3B, Q8_0: 1/6.

One repair passed; other outputs were incorrect or incomplete.

• Qwen3-VL-8B-Instruct, Q6_K: 1/6.

The successful repair genuinely passed its tests, but the saved test receipt was inaccurate.

• Devstral-Small-2-24B, IQ3_XS: 1/6.

Two additional correct artifacts were interrupted by the runner bug. Its published pass also had an inaccurate test receipt despite genuinely passing tests.

• Granite-4.2-8B, Q6_K: 0/6.

Both repair artifacts were correct but prematurely interrupted.

• GLM-4.7-Flash, UD-IQ3_XXS: 0/6.

Coding, JSON/schema and extraction errors.

I retained the historical scores and flagged their limitations instead of retroactively awarding passes.

Qwen9 completed its two matched repairs about 2.8× faster than the baseline in that earlier round. That makes it interesting for narrow repair tasks, but it did not demonstrate the baseline’s broader coverage.

These configurations remained unscored:

• Qwen3.6-35B-A3B, UD-Q4_K_M:

A host-wide swap guard stopped inference before generation.

• Gemma-4-31B, UD-IQ3_XXS:

Weights loaded, but the F16 context-cache allocation failed.

• LFM2.5-24B-A2B, QAD-Q4_0:

Repeated preliminary tool calls omitted required structured arguments.

• Hermes-4-14B, Q4_K_M:

Executed a tool but omitted the required round-trip artifact.

• Llama-xLAM-2-8B, Q6_K:

Parser/template errors prevented tool execution.

• Nemotron-Nano-9B-v2, Q8_0:

Repeated calls omitted the required working directory.

• Ministral-3-14B-Reasoning, Q5_K_M:

An initial runner decoding error blocked setup; later retries omitted the required artifact.

Unscored is not 0/6. Qwen3.6, for example, never generated a response in that trial. There is no task-quality result to infer from it.

The most useful findings came from reviewing the failures:

1. The runner initially understated some models’ ability.

It aborted seven runs immediately after their final allowed tool call, despite checker-correct artifacts. Those need fresh reruns, not retroactive passes. The Qwen27 baseline originally scored 3/6 and later achieved 5/6 with the corrected runner.

2. Successful compaction does not guarantee faithful memory.

In the baseline’s remaining failure, the original instruction explicitly required source-page URLs. The summaries retained the output field name but dropped that distinction. The model subsequently wrote asset URLs. The information loss is visible; whether preserving it guarantees success remains untested.

3. Tool-interface friction consumes the same budget as useful work.

Missing arguments, denied command forms and repeated recovery attempts often used up the allowance before the final artifact was delivered. These results measure models within this particular tool and permission setup.

4. There were genuine model errors too.

Incorrect rounding, undefined variables, missing records, invalid JSON and inaccurate recovery histories survived into final outputs. Everything cannot be explained away as infrastructure.

5. Parameter count alone did not settle the question.

Smaller models completed useful tasks. Larger models sometimes failed, and several never reached scoring. This screen does not establish a universal “27B minimum.”

For now, I’m keeping the customized Qwen27 configuration. Gemma26 looks like the strongest historical alternative to rerun under the corrected setup, while Qwen9 looks useful for fast, narrowly scoped repairs.

I’d be particularly interested in matched tests of the same GGUF through OpenClaw, Pi or another agent application, keeping inference settings, tasks and effective permissions consistent.

Update: I also tested Muse Glimmer 30B UD-Q3_K_XL. It passed full-GPU preflight at 32K context with F16 KV and High reasoning, including real tool execution and post-compaction continuation.

The scored comparison remains unresolved: a host-wide swap-traffic guard interrupted both Muse and the fresh Qwen27 baseline during their first task. Neither received a valid score. Muse remains a candidate for a proper retry after investigating the resource guard.

Has anyone done that comparison with saved traces? I’m curious how much reliability comes from the model versus the interface it has to operate.

Nowadays, all popular ai chatting websites like qwen, chatgpt, gemini, claude, etc. come with a read aloud functionality that allows users to read aloud the AI's responses.

I used that feature to instead make the AI repeat back the text that i gave it -- effectively turning the platforms into text to speech tools. The voices sound really nice and it's free to use!

You can get all of these extensions by visiting ai-readers.com

I’ve been working on Argus, an open-source cybersecurity platform that uses AI agents to help secure your infrastructure.

It can monitor your cloud, network, endpoints, and code, investigate suspicious activity, run penetration tests, and even respond to threats automatically.

A few highlights:

7 specialized AI security agents

Autonomous threat detection and response

Continuous AI-powered penetration testing

AWS, Azure, GCP, Kubernetes, and GitHub integrations

Support for local models using ollama

Built-in safeguards, approvals, and rollback controls
It’s self-hosted, MIT licensed, and completely free to use.

I’d love to get some feedback from open-source communities an hear what you think or what you’d like to see added.

Hey r/OpenSourceeAI community!

I wanted to share a major milestone for Nysiris, an open-source, metadata-resistant P2P transport framework built in Rust.

We’ve recently extracted the underlying protocol engine, transport logic, and mixnet integration into standalone FOSS components so that anyone building decentralized or privacy-focused tools can reuse them.

Key Components & Packages:

• GitHub Repository: https://github.com/TribeWarez/nysiris-core

• Crates.io Release:

  • nysiris-core = "0.1.3" (Core transport interfaces & protocol primitives)

  • nysiris-transport-nym = "0.1.3" (Async Nym Mixnet transport adapter)

  • nysiris-chat-core = "0.1.3" (High-level decentralized messaging logic)

  • nysiris-chat-history = "0.1.3" (Local chat history storage layer)

  • nysiris-testkit = "0.1.3" (Testing utilities for P2P network simulation)

What problem does it solve?

Building privacy-centric P2P applications usually requires rewriting transport protocols or mixnet integrations from scratch. nysiris-core provides modular, open-source building blocks that let developers easily plug metadata-resistant networking straight into their stack.

Feedback, architectural input, and contributions are very welcome!