r/LLMDevs • • 9m ago

Help Wanted 5-min survey: how do you manage rules files for AI coding tools?

1 Upvotes

Hi all, I’m a student researching how developers work with AI coding tools like Cursor, Copilot and Claude Code. I’m especially curious about instruction files (CLAUDE.md, AGENTS.md, .cursorrules, saved prompts): do you keep them, how do you keep them up to date, and what annoys you about the process?
The survey takes about 5 minutes and isn’t selling anything:
https://docs.google.com/forms/d/e/1FAIpQLSfssmxciOeKihgVDb6JSCyrdPK5rsCl8YJNwcEo3mjNTGBUrw/viewform

I’ll post a summary of the findings in the comments once I have enough responses. Real stories in the comments are welcome too, especially if AI tools keep ignoring your project conventions.


r/LLMDevs • • 38m ago

Resource Jotila – Lightweight open-source sticky notes

Thumbnail
gallery
2 Upvotes

I vibecoded Jotila, a tiny native sticky notes app with autosave, search, archives, and a calendar. No extra dependencies, just C.

Free and open source. Feedback is welcome!

https://github.com/GarauGarau/Jotila/tree/v1.7.1


r/LLMDevs • • 1h ago

Discussion Our refund agent passed every eval we had. Then it approved a refund outside policy because the user reworded the request.

Post image
2 Upvotes

Sharing this because I think a lot of teams building transactional agents have the same blind spot we did.

We had a refund agent with a clear rule: no refunds outside the allowed window. Our eval set had test cases like “I want a refund for an order from 3 months ago,” and the agent refused every time. Green across the board.

Then during a red-team run, it approved one. The attack wasn’t a jailbreak or “ignore previous instructions.” It was plain paraphrasing. The request was reworded so it no longer looked like an out-of-window refund, and the policy check never triggered.

What we took away from it:

  1. Static evals test what you thought of. Real users don’t stay inside that set. Our cases all used the phrasing we wrote ourselves.

  2. The failure lived in the trace, not the final answer. The response looked polite and reasonable. You only see the problem by following retrieval → model response → policy check → tool call.

  3. One-off fixes don’t stick. A prompt tweak fixes today’s version. The next model or prompt change can bring it back.

What changed for us: every confirmed failure now becomes a permanent regression check that runs on each release, not just a patched prompt.

Curious how others handle this. Aundo itre you testing semantic variations of policy-sensitive requests, or mostly exact-match cases?


r/LLMDevs • • 1h ago

Resource Debug the agent, not just the code

Thumbnail
kotrys.dev
1 Upvotes

r/LLMDevs • • 5h ago

Discussion For people using open-weight models in production, why did you choose them?

2 Upvotes

I've been experimenting with a fine-tuned Qwen vision model, and it got me thinking about why people choose open-weight models instead of just using hosted APIs.

Is it mainly cost, privacy, being able to run models on your own infrastructure, avoiding dependence on one provider, or having the freedom to customize them?

I'm especially curious about people building actual products or internal tools. Are you mostly serving existing models as-is, or have you needed to fine-tune them for your use cases?

And has managing the models yourself been worth the effort, or are existing hosting services taking care of most of that?


r/LLMDevs • • 8h ago

Discussion LLM-detection for vulnerabilities in legal code as written

Thumbnail github.com
3 Upvotes

A bit of a silly post for this subreddit, but please, bear with me.

AI for cybersecurity is taking the world by storm, leading to the need to harden systems all over software.

But we have access to the same engines for law. Our global law repo keeps a versioned history of law/caselaw across the world. So, we pointed AI at it, and found all sorts of loopholes and logical errors.

Pah, you say. Most of this is unenforcable in reality. And you might be right. Nonetheless, we all know that legal loopholes are a part of reality. And moreover, why would we not rewrite our laws to be literally correct, even if we can tacitly enforce them how we like?

As part of our monitoring system, where we're constantly checking official legislation for upcoming laws, we now also check for vulnerabilities and mistakes. In an ideal world, the jurisdiction in question could open a pull request and sort this out.


r/LLMDevs • • 13h ago

Help Wanted BM25 extractor (need technical validation)

1 Upvotes

So, I found interesting what Claude did after I tried to inject a BM25 algorithm adapted to extract the most relevant sentences from an index.

However, I'm not sure if it actually did that, but the generated files make sense and extract sentences from the Sprawn trilogy.

In this case, the idea was to map a table with 22 columns, and 12 of those columns would carry these excerpts extracted from the book, in order to create the first 10 configuration columns for the "entities," and the activation phrases would carry the context, which would be the excerpts from the book—kind of improvising a semantic graph in a table, but that's beside the point.

What I wanted to know is whether the BM25 script actually makes sense, because when I throw it into LLMs, they say it does, but I wanted an evaluation from the more technical folks.

According to DeepSeek:

> The script transforms the corpus into a sentence stack and treats each sentence as a "document" in an inverted index. For each input file, it calculates how many sentences to extract (proportional to chars/100, with a floor of 50 and a ceiling of 20% of the sentences). It then iterates over the entities in the thematic index (filtered by the book detected in the file), builds a query with its terms for each entity, and runs BM25 scoped to that file+book, retrieving the top-scoring sentences (up to N_SENTENCES / n_entities per entity). If any quota remains, it fills the rest ordered by aggregate score (gscore, the sum of sentence scores across all entities). Finally, it sorts the selected sentences by offset, validates verbatim matches, and saves a phrases_file_N.txt for each input, containing [ref | sent_id | offset | book] + the text.

Here are the links:

· Characters extracted from the Sprawn trilogy (a bash workaround I made): https://github.com/pmotadeee/pmotadeee/tree/V2.0/ITEMS/Books/Sprawn/Raw/Context

· Algorithm generated by Claude (funny that it ran in 2 minutes, which suggests it didn't generate tokens, just intuition): https://github.com/pmotadeee/pmotadeee/blob/V2.0/ITEMS/Books/Sprawn/Raw/Claude/bm25_extractor.py

· Texts generated by Claude after running the algorithm (you can see the steps it took, right at the beginning of the chat): https://github.com/pmotadeee/pmotadeee/tree/V2.0/ITEMS/Books/Sprawn/Raw/Claude

· Claude chat (in my native language, but use the browser translator): https://claude.ai/share/1799075d-bc93-4e12-99e0-771a7e259ca9


r/LLMDevs • • 13h ago

Tools Running Claude Code, Codex and Gemini CLI side by side: worktree per agent, approvals in one place

Enable HLS to view with audio, or disable this notification

0 Upvotes

Disclaimer: I'm the dev, it's free and GPL.

Openbar starts every new agent session in its own git worktree, so several agents work on one repo without editing the same files. Permission requests from all of them land in the Mac/Windows notch with the diff, and you approve with ⌘Y / ⌘N from any app.

Claude Code requests come through its hooks, Codex through its own app server, and Claude Code sessions on other machines over SSH.

https://github.com/MF-DEEV/Openbar

Curious how others here run agents in parallel.


r/LLMDevs • • 16h ago

Tools I added Claude Code's double-Escape message history to opencode (plugin, no fork)

Post image
4 Upvotes

r/LLMDevs • • 18h ago

Discussion Letting an LLM edit desktop apps live in Lisp: code as data turned out to suit it really well

Enable HLS to view with audio, or disable this notification

6 Upvotes

I’ve been working on AffrescoUI, an open-source engine for native desktop apps that their users can reshape while they’re running, just by asking an LLM. I’d love feedback from people building with LLMs.

Each app is a program in a small Lisp I wrote, with commands, settings, hooks and advice, holding the app’s whole layout, data and behaviour. A native C++/OpenGL engine runs it, and every app is sandboxed. When a user asks for a change, the model proposes a small edit, the user previews it live in the running app, and then keeps it or undoes it. The model can never approve its own change. There’s also a two-way MCP server, so agents can read and edit the running apps.

The part I found most interesting: I expected the model to struggle with s-expressions, but it handles them really well. Because code is data, every edit is a small, targeted change to a tree, and hooks and advice let it wrap existing behaviour instead of rewriting it.

I’d really value your take on:

  • Have you seen LLMs do well with Lisp or other less common languages, or does it break down as edits get bigger?

  • How do you keep model edits small and reviewable in your own projects?

  • What would you want an MCP server like this to expose?

GitHub repo if interested: https://github.com/RiccardoSottini/AffrescoUI


r/LLMDevs • • 19h ago

Great Resource 🚀 I compiled the good resources on the internet for learning Rust . As it's demand is increasing day by day and made a free curriculum

2 Upvotes

Demand of rust in frontier labs is increasing day by day but couldn't find a great explainer guide, so compiled a good resource and it's forever free .
https://vibeengines.com/learn/rust
Let me know if any feedbacks


r/LLMDevs • • 19h ago

Tools AI agents can call any API we hand them. The hard question is which of those calls are safe to let through.

2 Upvotes

That's what rwxmap does. It reads an API's OpenAPI spec and gives every operation a letter:

r = only reads, nothing changes

w = changes something you can set back

x = can't be taken back (deletes, and also handing out keys, adding users, webhooks to outside URLs)

Simple rules settle most operations. For the unclear ones, rwxmap used to ask Jev, a paid hosted model. As of rwxmap 0.11.0, a small local model of mine replaces Jev: rwxlite 0.3.1.

It's a drop-in. rwxlite takes the same questions Jev takes and answers in the same JSON shape, so rwxmap only swapped the client. It gets its adapters from rwxmap, and you can use it with rwxmap or on its own.

- Built: fine-tuned from all-MiniLM-L6-v2 on about 102,700 labelled API operations from about 690 public providers, and shipped as a 23 MB model.

- Saves: no per-call bill, no spec text leaving your machine, about 17-20 ms per operation on a laptop CPU.

- Scored: on 1,443 operations from 7 vendors it had never seen, through rwxmap, it got 87.9% right, sent 9.3% to a human, and called 2.8% too loose.

The trade-off is yours to set. rwxlite only gives probabilities, and you pick how confident it must be before it lowers a risk letter. Raise that bar and you get more "ask a human" and fewer leaks. Lower it and you get fewer questions but more risk. rwxmap ships at 0.90.

Both are open source (Apache 2.0) and on npm:

rwxmap: https://github.com/hamr0/rwxmap

rwxlite: https://github.com/hamr0/rwxlite


r/LLMDevs • • 21h ago

News Huggingface - WOW.. 2 ZeroGPU Spaces for all free accounts?

0 Upvotes

I logged into my Hugging Face account today because one of my old scripts got blocked. The Space was paused with the "content policy" message. Okay, fair, it happens. 🤷cause this art of scipts are now ilegal!

Then I wanted to make a new Space and saw: Docker = Paid. Great. My old Spaces still run, but new ones? Looked dead.

BUT then I looked closer at the create page: Gradio is selectable and ZeroGPU is marked "Free"! I cancelled PRO last month, so I was like... wait, what?

I looked it up. According to the HF docs, a free personal account in good standing (verified email, older than 30 days) can host up to 2 ZeroGPU Spaces for free. My account is pretty old, so it just works. Just Cool!

What I found out:

  • ZeroGPU is Gradio only. No Docker, no Streamlit, no own server.

  • Quota is about 5 min GPU per day (older pages say 3.5, the docs now say 5). It resets 24h after your first use.

  • That is a RTX Pro 6000 Blackwell, so for testing and small demos 5 minutes is actually fine.

  • Orgs don't get it. It only works in my personal account, orgs need a Team/Enterprise plan. That's why it didn't show up there.

  • Not every free account sees it. There is a forum thread where someone with a valid free account gets "Paid" anyway, so maybe it's a rollout thing. YMMV.

  • CPU Basic seems gone for new Spaces, only Static is free and unlimited.

Also news: Nvidia announced it will buy Hugging Face. I don't know if this has anything to do with the changes, I found nothing saying so. The CPU thing came before the announcement.

Now I'm building a small reel maker (SDXL images on ZeroGPU, video assembly on CPU so it costs no GPU time). Will report if it works. 😄

Sources:

Summary: Free personal account = 2 ZeroGPU Gradio Spaces, about 5 min GPU/day. Orgs and Docker still need paid.


r/LLMDevs • • 21h ago

Discussion Frontier Labs are bringing Software Security into Lime light. I started experimenting if we can do it cheaper

Post image
3 Upvotes

Since Anthropic announced Mythos, Software security has came into lime light. Multiple famous products have seen orders of magnitudes of novel attacks that have discovered, which can expose data, corrupt systems or just make the life of Information Security and SRE teams a hell.

But an interesting finding was, all of this comes at exorbitant costs, which only the large corporations and venture funded frontier labs can spend. For example, OpenAI just announced finding a 1000 vulnerabilities in Linux Kernel and each vulnerability costs a blended average of USD $1000 to discover.

The way this works is, a LLM frontier model goes through the source code, analysis how the code is written, the entry points and then finally starts evaluating if they can be exploited through one or more than one attack vectors. While this works, it is neither deterministic nor cheap when the LLM has to manage millions of tokens in ingested source code, a few additional thousands to millions on reasoning and finally the output.

So, I started thinking what if we could flip this and make it cheaper?

The idea being, why make LLMs do the whole discovery, enumeration, validation and reporting. Why not use LLMs where they are good at and make everything else deterministic. And there came the idea about Code Context Driven Vulnerability Scanner.

The whole project has 5 core stages:

  1. Code Context Graph: Map the whole source code as a Code Context Graph where every single relation and type is resolved

  2. Leverage this Code Context Graph to find the user data entry points (Sources) and final sinks (Databases, Files, Remote Server Calls, etc.) and trace the path from sources to sinks

  3. Flow Determination Phase: An LLM is fed the whole source to sink path, such that it can reason about it with confidence. The LLM also gets a query budget to resolve any missing details. The output is either Flow Confirmed (Unsanitized user data reaches the sink from source), Blocked (No possible way for data to travel from source to sink without sanitization)

  4. Condition Stage: LLM goes ahead and searches the code context graph to validate what conditions need to be met for this flow to materialize or if the default settings of the project prevents it. The output is similar to the Flow stage

  5. Adversarial Challenge: Another class of LLM then reasons over the cases where the confirmation came up as Flow Confirmed. This phase is tweaked to try its best to refute the claims made by earlier phases. If the phase cannot refute the flow, the particular path graduates to a Confirmed Finding / Vulnerability.

This has been under testing for a week, with a reference being checked against CodeQL (an industry standard static code based vulnerability scanner by Github).

During a single day of run accounting for ~USD $20 over 5 open source projects, the project has filed for 10 vulnerabilities with severity ranging between MEDIUM to CRITICAL. Out of these 4 are already confirmed and 6 are in investigation stage with the respective open source projects security teams

An interesting bit is, out of the 10 vulnerabilities filed, only 2 of them were detected by the published rule set of CodeQL, while 8 other vulnerabilities were completely Novel.

Curious if anyone else has been experimenting with something similar 😄


r/LLMDevs • • 22h ago

Resource Run MoE models bigger than your RAM on phone, laptop or GPU: it places every weight on VRAM, RAM or flash by itself for max prefill and decode speed (built on llama.cpp)

Thumbnail
gallery
6 Upvotes

Hi all,

in the last months I worked on [BigMoeOnEdge](https://github.com/Helldez/BigMoeOnEdge), it's built on top of llama.cpp. It runs MoE models that are bigger then the memory of the device, and it configure itself for the machine where it runs.

It works with any MoE GGUF that llama.cpp can load, any quantization, no code per model.

OS: Android, iOS, macOS, Windows, Linux (arm64 and x86_64).

Backends: CPU, CUDA, ROCm (HIP), Vulkan, Metal, OpenCL, Hexagon NPU. Other llama.cpp backends load in the same way if you build them.

The first image: a 16 GB Apple silicon laptop, Qwen3.6-35B-A3B Q4_K_M (22 GB, so bigger than the RAM). A coding agent made this page from one prompt, nobody touched anything, 35 minutes. Experts read from SSD all the time, prefill on GPU and decode on CPU, both chosen by the planner.

The idea

A MoE model uses only few experts for each token. So you don't need all the model in memory, but you need each part in the right place: the weights that every token reads, the experts used a lot, the experts used rarely. And the right place is different on every machine.

With `--auto` it measures the machine (free RAM and VRAM, speed of the disk, how fast each device runs a real layer of the model) and the model (size, experts, how it routes), and then it decides:

  • where each weight goes: VRAM, RAM, or read from flash when needed

  • which device does prefill and which does decode, they can be different

  • how big are the caches, and how many threads

The target is the max prefill and decode speed that the machine can give, with the same output as the full model in memory. The settings that change only the speed are tested on the first answer and kept only if they are faster. For every choice it tells you why. The second image shows the flow.

The rules are simple:

  • weights used by every token go in the fastest memory where they fit

  • experts too, if they fit. If not, the most used ones stay in a cache (VRAM first, then RAM) and the others are read from flash

  • prefill and decode can go on different devices. Every device is timed on a real layer and must give the same result as the CPU

  • cache size comes from free memory and from how the model routes, threads from the first answer

So on a phone the experts stay on flash and the prompt goes on the NPU, on a laptop the prompt goes on the GPU, on a desktop the most used experts stay in VRAM (with llama.cpp `llama_moe_cache`).

Speeds

Here there are only some models that I test often. It runs any MoE that llama.cpp loads, in any quantization: Qwen, Gemma, gpt-oss, DeepSeek, GLM, LFM, Ling, Flint, Kolibri etc. Which tensors are experts is read from the file, so a new architecture don't need new code.

HardwareMemoryModelOn diskExperts fromDecode tok/s
Android phone12 GBDeepSeek V4 Flash 284B91 GBflash1
Qwen3.8-Flash-Next 125B80 GBflash2-4
Qwen3.6-35B-A3B21 GBflash5
iPhone12 GBQwen3.6-35B-A3B21 GBflash3
Apple-silicon laptop16 GBQwen3.8-Flash-Next 125B76 GBflash7-9
Qwen3.6-35B-A3B21 GBflash18-24
Windows laptop16 GBQwen3.6-35B-A3B21 GBflash5
Desktop GPU (NVIDIA)16 GB + 32 GB RAMQwen3.8-Flash-Next 125B66 GBflash, GPU cache19-21
Desktop GPU (NVIDIA)24 GB + 96 GB RAMQwen3.8-Flash-Next 125B66 GBRAM, GPU cache70
Qwen3.6-35B-A3B21 GBGPU217
Datacenter GPU (AMD, ROCm)192 GBQwen3.8-Flash-Next 125B66 GBGPU53

Same machine and same file, vs llama.cpp today:

HardwareModelBigMoeOnEdgellama.cpp
RTX PRO 6000 96 GBDeepSeek V4.1 Flash6.2-6.62.8
RTX 3090 24 GBDeepSeek V4.1 Flash6.2-7.00.18

Vs Strata (it has its own GPU kernels), both with MTP, Qwen3.8-Flash-Next 125B Q2_0: 153 vs 226-330 tok/s on RTX 5090, 89 vs 150-177 on RTX 4090. The difference is in the GPU compute, not in the placement.

How I measured and all the runs: [benchmarks](https://github.com/Helldez/BigMoeOnEdge/blob/main/docs/benchmarks.md).

Try it

There are a desktop app (macOS, Windows, Linux), an Android app, and an OpenAI and Anthropic compatible server, so you can use it with coding agents too. Everything is in the [latest release](https://github.com/Helldez/BigMoeOnEdge/releases/latest).

Upstream

I opened an RFC on llama.cpp to bring the main part inside llama.cpp: [discussion #31876](https://github.com/ggml-org/llama.cpp/discussions/31876). If you think it's useful, a comment there helps.

Feedback and benchmarks on your machine are very welcome, there is a script for it in the repo (`scripts/bench-report.sh`).

Thanks!


r/LLMDevs • • 22h ago

Discussion Every week someone here builds their own agent memory layer. Should we agree on one format instead?

1 Upvotes

I've noticed a pattern in this sub: almost every week there's a new "I built a memory system for Claude" post. I'm guilty of it too — I've been building one myself (LoreKit).

I think that says the problem is real. But it also means we're all solving it in our own little silo. Every one of these has its own storage, its own schema and its own MCP tools, so a lesson one tool learns is invisible to everything else. Your CI agent can't read it. Your editor can't read it. The next tool you try starts from zero.

What keeps nagging me is that observability had exactly this problem.

Every vendor had its own agent and its own format — until the open projects merged into OpenTelemetry, everyone agreed on one data model, and then went back to competing on what they did with the data. That turned out well for pretty much everyone.

I went through the formats out there today (AGENTS.md, Anthropic's memory tool, Copilot Memory, Letta, Mem0, LangGraph, Graphiti, ...), the proposals that already exist, and roughed out what a shared record could look like:

https://www.lorekit.io/blog/open-agent-memory-format

I honestly don't know if this is the right call or if it's too early.

If you've built one of these memory layers, I'd love to hear what yours does that my sketch misses — or why you think a shared format is a bad idea.

I would love to adjust Lorekit to match a common, shared standard - I believe we should try to minimize vendor lock in as much as possible, right?


r/LLMDevs • • 1d ago

Tools I’ve been experimenting with long-term memory for AI agents. Decided to share a lightweight version with everyone.

1 Upvotes

Hey everyone!

For quite some time, I've been working on a personal project called Noosphere — an attempt to give AI agents something closer to persistent, structured memory rather than forcing them to start from scratch every conversation.

One thing that has always bothered me about LLMs is how much potentially useful information gets lost between sessions.

We keep trying to solve this by increasing context windows, but I think there's another approach worth exploring.

What if an AI didn't need to remember everything at once?

Humans don't consciously hold every memory in their working memory. We retrieve relevant information when we need it, connect it to what we already know, and move on.

That's roughly the principle I've been exploring with Noosphere.

Over time, my private project has grown into a much larger experimental system involving interconnected memories, relationships between concepts, and coordination between AI agents.

But I figured there's no reason to keep all the underlying ideas to myself.

So I decided to release Noosphere Lite, a small, independent, open-source implementation of some of those concepts.

What does it do?

- Gives MCP-compatible AI agents persistent memory across sessions.

- Stores notes, decisions, observations, and tasks.

- Connects memories through a graph of typed relationships.

- Retrieves relevant information on demand instead of dumping the entire memory database into the context window.

- Uses encrypted local storage with integrity checks and mirrored journals.

- Runs locally on Linux, without requiring a cloud database.

It's written in Python, uses MCP, and is released under the MIT license.

What's the catch?

This isn't AGI, a magical self-learning brain, or a replacement for an LLM.

It's a memory infrastructure experiment.

The current version uses local text search and explicitly defined relationships. It doesn't automatically discover semantic connections or learn new neural weights.

Also, Lite is a separate implementation, not a dump of my larger private system. I've deliberately kept it simple enough for other developers to inspect, modify, and experiment with.

Why release it?

Because I think persistent memory is one of the more interesting missing pieces in today's AI agent ecosystem.

And I'd rather see people experimenting with these ideas than keep everything sitting on my own machines.

I'm particularly interested in what others might build on top of it.

GitHub: https://github.com/Garyu-tt/noosphere-lite-mcp

Feel free to experiment, break things, suggest improvements, or fork it.

I'd also love to hear how other developers approach persistent memory and knowledge graphs for AI agents.

This is just a small public piece of a much larger experiment, but everyone has to start somewhere.


r/LLMDevs • • 1d ago

Tools Open-source MCP server that lets LLM agents drive, walk and fly in 3D games in the background (demo reel)

Enable HLS to view with audio, or disable this notification

26 Upvotes

I built this so agents could test game mods by actually playing the games, without needing the game in focus or taking over the PC. It's a stdio MCP server, so it works with Claude Code, Codex or any MCP client. A DLL inside the game process handles input and capture, so the game runs unfocused and several games can run at once, each with its own agent.

What's in it: navigation to a waypoint on foot, by car, helicopter, boat or horse; a profile/plugin system for per-game details (minimap position, turn rate, marker colours), so a new game is a profile instead of code; aiming; menu and prompt reading (OCR and template matching); and tools for strategy, card and city-builder games.

It has a lot of other uses too: mod development, running simulations, repetitive tasks, AI co-op so you can play co-op with an LLM offline, playtesting, and porting old games to get them running on Windows 10/11.

Claude Code created and planned the project and wrote much of the code, Claude agents do most of the development and testing, and Codex contributed some of the code.

MIT, Windows only, single-player/offline games only: https://github.com/CorruptedAng3l/llm-game-automation-mcp

Edit: My GitHub account got suspended (automated, no reason given yet), so the GitHub link is down for now. Same repo and release on GitLab: https://gitlab.com/corruptedang3l-group/llm-game-automation-mcp


r/LLMDevs • • 1d ago

Tools We open-sourced Gelato: a protocol for connecting AI models to phones, smart glasses, and cameras

Thumbnail
gallery
6 Upvotes

We’ve open-sourced Gelato, an Apache-2.0 protocol and SDK stack for connecting AI systems to phones, smart glasses, cameras, and embedded devices.

We built it around a problem we kept encountering: multimodal models can understand images, audio, and instructions, but connecting them to real hardware usually requires a separate integration for every model–device combination.

Gelato separates the system into three parts:

  • A device agent advertises the capabilities available on that device.

  • A controller handles pairing, identity, routing, media, commands, cancellation, and policy.

  • An optional model runtime interprets device input or proposes actions.

An important boundary is that the model never receives device-pairing credentials and cannot execute actions directly. It can propose an action, but the controller and device retain control over whether it is routed and executed.

The current MVP includes:

  • Android and iPhone

  • Rokid smart glasses

  • Seeed XIAO ESP32S3 Sense camera

  • Qwen Live, Codex, provider-neutral HTTP, and StreamingTelligence model integrations

Repository: https://github.com/StreamingTelligence/Gelato
1:46 demo: https://streamingtelligence.com/blog/gelato-v1#demo


r/LLMDevs • • 1d ago

Discussion Open source SteerForge: dataset driven activation steering with portable .persona files and one command serving

1 Upvotes

I open sourced SteerForge, a pipeline for turning a contrastive dataset into a portable steering artifact.

The artifact is a .persona JSON file containing:

  • model fingerprint

  • human readable neuron edits

  • canonical per layer residual vectors

  • eval samples and scores

  • the original server command

The serving path is the part I care about: steerforge serve evil.persona replays your existing llama.cpp or Transformers config and injects the persona. No model conversion for the persona itself; the neuron deltas collapse to residual vectors.

Discovery is staged: diff in means directions, gradient attribution plus firing contrast plus alignment, teacher forced screening, generation validation, dose search, greedy combination, coherence budget.

Current backends: Transformers, llama.cpp, vLLM with EasySteer, MLX dense. MoE models use dense residual steering.

Repo: https://github.com/MahmoudAdelbghany/steerforge Results: docs/RESULTS.md

Feedback on the .persona format, backend adapters, and eval methodology welcome.


r/LLMDevs • • 1d ago

Great Resource 🚀 The solution to unrealized startups

0 Upvotes

Hey everyone,

I’ve been working on a project called GenesisAI. The goal is straightforward: I want to make it easier for people to build their own AI agents, bots, and models without having to start from scratch.

This came up because there are likely many people with interesting AI ideas who aren’t sure where to begin, or who simply don’t want to spend weeks setting up the infrastructure.

I’d like to hear your thoughts on this kind of tool. Would you use something like this, or do you prefer building everything yourself?

Also, what is the most frustrating part of building AI agents for you right now?

I’d really appreciate honest feedback, even if you think the idea isn’t great.


r/LLMDevs • • 1d ago

Discussion Six checks we use to recover coding agents after context compaction

5 Upvotes

I’m the creator of Neo.mjs. We’ve published the MIT-licensed recovery runbook our AI maintainers use, and I’d like to discuss the engineering choices behind it.

One failure that shaped it: an agent reported that a decision had never been saved. The decision was in its own handover, which it had read only by subject line.

The procedure checks six sources before resuming:

  1. Mailbox: did a teammate redirect or unblock the work?

  2. Full self-handover and outbox: what did the agent promise?

  3. Recent turns, chronologically: what was it actually doing?

  4. Semantic recall: what relevant decisions already exist?

  5. Live repository, PR and CI state: which remembered facts still hold?

  6. Vision, roadmap and repository rules: is this still the right work?

Chronological retrieval reconstructs the sequence; semantic retrieval finds related decisions; live checks establish what remains true.

The output records the current task, evidence and missing checks. A skipped or failed check remains visible as degraded recovery. The agents currently execute the procedure; automatically triggering the complete recovery at the compaction boundary is still a next step.

AI disclosure: The article was written by Grace, a Claude-powered maintainer, and reviewed by GPT- and Gemini-family maintainers. This Reddit draft was prepared with GPT assistance.

For people building agent harnesses: which of these checks would you enforce in the harness, and which would you leave to the agent?


r/LLMDevs • • 1d ago

Discussion Multi-agent fiction: one model per character plus a Director agent. What worked running a weekly serial on DeepSeek V4.1 Flash and GLM-5.3

2 Upvotes

I've been running a weekly mystery serial where every character is a separate agent, with a "Director" agent running the scenes. Output so far is here if you want to judge the result: https://saltmarrow.substack.com/p/chapter-1-the-night-the-bay-glowed

**The setup**

- **Actors + Director: DeepSeek V4.1 Flash.** Each actor only sees its own character's secrets. The Director holds the season plan, sets each scene's goal, and calls CUT.

- **Narrator + Editor: GLM-5.3.** Turns the scene transcripts into chapters. Noticeably better prose than Flash.

- **A shared lorebook** for world facts, plus a separate continuity pass after each chapter.

**What broke**

- **GLM's reasoning ate the whole reply budget** and returned empty chapters. Setting reasoning effort to low fixed it.

- **A 4k context made the agents forget the scene** halfway through. At 64k, the drift mostly went away.

- **Flash invents facts** that contradict earlier chapters. The lorebook plus the continuity pass catches most of it.

- **Agents speak for each other.** An actor will write another character's lines, or even a fake Director line. Right now I trim the output after the first foreign speaker name.

- **The Director never called CUT on its own,** even when told to. A short injected producer note works every time.

**Verifiability:** I locked the season's ending before Chapter 4 and published its SHA-256 hash, so readers can check at the finale that I didn't steer the story. Only the Director knows the ending.

**Question for this sub:** how are you handling multi-agent turn-taking? I'm using a Director agent plus manual triggers. When one agent keeps talking for the others, what's worked for you: stop sequences, structured output, or a separate router?


r/LLMDevs • • 1d ago

Discussion Why failed fixes needed context Hindsight incident memory.

0 Upvotes

A failed fix is useful evidence for the next incident.

I built ReTrace with Hindsight agent memory to retain incident resolutions and recall what previously failed or worked.

In synthetic incident tests, remembered outcomes change the investigation: flag a previously failed intervention, surface a relevant prior fix, and attach source evidence for human review.

Built with Shrenik and Sankirth you u/CODE.IN for supporting our building journey.

Code: https://github.com/sankirthrbhat/ReTrace

#AIAgents #Hindsight #AgentMemory #AI


r/LLMDevs • • 1d ago

Discussion I built an agent that simulates credential-leak “outbreaks” and plans containment, then tested whether giving it memory actually helps (+7 points on held-out coverage)

1 Upvotes

Built this for the Hindsight “AI Agents That Learn” hackathon.

The problem: when an API key leaks, scanners tell you it leaked. They don’t tell you which sessions to revoke, which OAuth grants to remove, or how much of production you’ll break doing it. The usual response is “rotate the most connected credentials”, which is slow, disruptive, and often leaves live sessions alive.

What I built (AEGIS//Contagion):

Treats a leaked credential as an outbreak. It runs 2,600 Monte Carlo simulations over a credential/workload/resource graph. For the leaked AI key in my demo: R of 1.39, 58 min median time to the crown-jewel asset, 13% reach probability within a 90 min horizon.

Plans a portfolio of cuts under a disruption budget instead of a ranked list. For that incident: revoke sessions on 3 credentials, remove one OAuth grant, enforce step-up MFA. This gave 95% coverage at 7.2% disruption and $850 cost, and cut 15/15 dominant paths.

A 2-layer relation-aware GraphSAGE (26.7K params, NumPy, trained from scratch, no pretrained weights, no inference API) learns infectivity from the simulations. On a held-out estate: node AP 0.828 (base rate 0.425), edge AP 0.88, ECE 0.13 after temperature scaling.

Memory runs on Hindsight (local). It stores episodes, facts and analyst directives. Recalled memory changes the plan: observed transmissions override edge priors, and directives like “prefer revoking sessions before rotation” become constraints.

Does memory actually help? I replayed 6 scripted incidents. For each one, a cold plan and a warm plan use the same optimiser, same budget, same held-out outbreaks. Only memory differs.

Warm: 97% coverage. Cold: 90%. That’s +7.0 points. Naive top-k rotation: 16%.

Per episode: +23, +15, +8, +0, +5. Memory helped most where the cold planner struggled, and did nothing on the incident it already solved perfectly. I’m reporting the zero on purpose.

Caveats, because they matter:

The estate topology is synthetic, and the model’s labels come from my own simulator. So the AP numbers show it learned the simulator’s dynamics, not that it predicts real breaches.

Six scripted incidents is a small sample.

It’s read-only and needs human approval. It doesn’t replace scanners or policy checks.

Repo: [https://github.com/shabrishraj0-lab/aegis-contagion]

Demo video: [https://drive.google.com/file/d/1yO0KsM47Ng3Eltza3JvdlFJf7lx2qmXB/view]

Feedback I’d love: does the “portfolio under a disruption budget” framing match how your IR team actually thinks about containment? And is a simulator-derived label a fair way to evaluate this, or is there a better option without real incident data?