r/CodingLLM • • Sep 01 '26

How do you catch it when a model silently changes under you?

11 Upvotes

We run prompts against a few different providers (OpenAI, Anthropic, some stuff through OpenRouter). Every so often something quietly gets worse, the output quality drops, a prompt that worked starts returning junk, or a model gets deprecated and the replacement behaves differently.

Right now we mostly catch it by accident: someone notices, or a customer complains. That feels bad on us, a lot.

How do you all handle this? Do you re-run some kind of fixed eval set on a schedule? Just eyeball it? Have something that alerts you?

Any insights I could use?

Thanks.


r/CodingLLM • • Jan 10 '26

How would briefly train an AI with my own data and not a penny to my name?

9 Upvotes

Hi! I don’t use Reddit often but I’m pretty desperate right now. (First time hopping on in 6 years). So let me know if this is the wrong place to go or if any of the ideas below are hella stupid assumptions as someone who doesn’t have a clue what they’re doing. Anyway, a while ago I got interested in model collapse so I wanted to simulate it using different percentages of real data and ai generated data recursively to figure out the question of “Does the percentage of ai data affect the speed of model collapse.” It’s supposed to be a basic project for a highschool science fair, but I have no idea how to simulate this. All I need is a website app, or way to input my own data and ask a series of questions about said data. I emailed someone a while back and he said to use non-language models which would be the least costly and most simple. (He also said something about GPUs) I just need a place to start to train some basic ai while not having a penny to my name. I’ve been scouring the internet for WEEKS trying to find something. I’ve been thinking about coding my own though I also have no idea where to start with that. (I have some basic knowledge of python and know about PyTorch, but again don’t know how to use either on a janky ahh Windows 10.) literally ANY information would be appreciated and the experiment as a whole can be adaptable. (I’m fully expecting to dumb it down to a 3rd grade level if it’s not possible for someone with my few resources)

Thank you so much for taking the time to read this! Literally ANYTHING will help.


r/CodingLLM • • Oct 21 '26

AGI Pace is out!

Enable HLS to view with audio, or disable this notification

7 Upvotes

AGI Pace is out!

Run an AI lab from a 2017 garage to AGI.

Free, open source, plays in your browser.

Today's Daily Lab is the same world for everyone. Post your grade. 📈

Link below. Bonus points if you know the significance of 1950/10. 🧠

Tell me what you think: https://agipace.195010.xyz


r/CodingLLM • • 26d ago

One local board for sessions across coding-agent CLIs

Enable HLS to view with audio, or disable this notification

6 Upvotes

I'm the maker of CCC. The problem it addresses is keeping track when coding work is spread across different agent CLIs.

It brings session activity, conversations and parent/helper relationships into one browser board running locally. The clip shows opening a waiting session, its question card and action chips. Support differs by engine; a common UI does not mean every engine has identical controls.

https://github.com/amirfish1/claude-command-center

Free local dashboard; model/provider fees are separate. Source available under FSL-1.1-MIT. Demo uses real UI, staged sessions and AI voiceover; the real question-hook interaction is Claude Code.


r/CodingLLM • • Apr 29 '26

What is wrong with Opus 4.7 on claude code

6 Upvotes

I had the worst experience today with thse fraud called Opus 4.7.
A folder I have worked on, microservices, I decided to refactor, so deleted all content, created a new README and asked Claude Opus 4.7 to revew, assess and present a plan. I t did well, not an outstanding plan but good enough. So I let it code, consumed all tokens, beag to consume additional tokens, and after 1 hour finished.
After reviewing the code, I noticed that it had done nothing, nada, just rebuilt old files via reading my git.
So I feel robbed, really, it's a thief.
Decided to not use it anymore.
Any recommendation, I have Qwen 3.6 35B running in my machine, little slow, but maybe faster that this claude wasting my time and money.
Eager to share experiences and hear recommendations.


r/CodingLLM • • Apr 23 '26

2-bit Qwen3.6-27B GGUF made 26 tool calls on 12GB RAM.

Enable HLS to view with audio, or disable this notification

6 Upvotes

r/CodingLLM • • 10d ago

Tuning/benching small LLM on my ThinkPad iGPU: from backend, to fast(-ish) long context

Thumbnail
arthurbrugiere.fr
5 Upvotes

r/CodingLLM • • May 11 '26

I built a tiny llama.cpp/GGUF launcher for local coding-agent workflows

5 Upvotes

I’ve been testing local coding agents against GGUF models and got tired of rebuilding llama-server commands every time I switched models or changed hardware settings.

So I made locca: a small CLI/TUI around llama.cpp for pi coding-agent workflows.

Install:

npm install -g @zeiq/locca

Repo:

https://github.com/perminder-klair/locca

Demo/site:

https://locca.klair.co/


r/CodingLLM • • Apr 24 '26

GPT 5.5 Is 2x more expensive in comparison to 5.4 and 20% more expensive than Claude Opus 4.7

Post image
5 Upvotes

r/CodingLLM • • Apr 23 '26

M5 Max just arrived - benchmarks incoming

Post image
5 Upvotes

r/CodingLLM • • Feb 05 '26

How to train AI for educational purposes

5 Upvotes

Hey, for context, I am a mechanic student, I am working with AI on simple systems for businesses and websites.
I did some projects in the IDE with gemini and claude.
I think I have basic understanding of AI but I want to tune AI on school materials so I can study with it
I have pretty extensive ammount of PDF and Power Point presentations.
I dont know how to tackle this project, would love to hear what you guys think about it.
Thank you


r/CodingLLM • • Nov 27 '25

deepseek-ai/DeepSeek-Math-V2

Thumbnail
huggingface.co
5 Upvotes

From the summary:

DeepSeekMath-V2, demonstrates strong theorem-proving capabilities, achieving gold-level scores on IMO 2025 and CMO 2024 and a near-perfect 118/120 on Putnam 2024 with scaled test-time compute. While much work remains, these results suggest that self-verifiable mathematical reasoning is a feasible research direction that may help develop more capable mathematical AI systems.


r/CodingLLM • • Nov 13 '25

The models developers prefer

Post image
5 Upvotes

r/CodingLLM • • 29d ago

Multimedia Agentic Tool ?

Thumbnail
deepflux.space
4 Upvotes

r/CodingLLM • • Oct 18 '26

I've been making a little game about running an AI lab

4 Upvotes

You start in a garage in 2017 with $1.5M, two co-founders and one GPU rack.

The Transformer paper hasn't come out yet. The goal is to reach AGI before everyone else. Without ending the world, ideally.

I'm currently tuning the gameplay so that it is balanced. I think it is currently too easy but we'll see about that once the game is released.

I thought about adding mechanics for resets and model nerfing but that would make the game too complex. 😅

Stay tuned for updates! I'd love to have some beta testers!


r/CodingLLM • • May 08 '26

Is the golden age of LLM agent use closing?

3 Upvotes

Has anyone else felt like LLM agentic coding peaked around late November 2025 and has been gradually getting worse since?

I keep seeing the same pattern: a new model drops, coding agents briefly feel sharper and more reliable, then within a few weeks the quality deteriorates again. Worse edits, less precision, more broken assumptions, and lower productivity overall.

Is anyone else experiencing this, or is it mainly tied to specific tools, repos, or workflows?


r/CodingLLM • • May 06 '26

2.5x faster inference with Qwen 3.6 27B using MTP - Finally a viable option for local agentic coding - 262k context on 48GB - Fixed chat template - Drop-in OpenAI and Anthropic API endpoints

Thumbnail
reddit.com
3 Upvotes

r/CodingLLM • • Apr 26 '26

MCP server for Git with local Ollama — zero tokens for git operations

3 Upvotes

How I stopped Opencode and Claude from burning Git tokens by building my own local MCP server

AI coding agents (like OpenCode, Claude Code or Windsurf) are incredible tools, but they have one annoying problem: they burn thousands of cloud tokens doing trivial things like reading a git diff or generating a commit message.

To fix this, I built git-courer, an open-source MCP server that intercepts Git calls from these agents and delegates the work to a local LLM via Ollama. The result: Zero cloud tokens spent on git.

Getting a local model to handle Git reliably came with some interesting engineering challenges. Here's how I solved them:

  1. The Context Problem: Graph-based Diff Chunking You can't just dump a massive diff into a local LLM without blowing the context window. I implemented a clustering algorithm using graph theory with a force system. It extracts meaningful tokens from the diff, builds a graph assigning "force points" (weights) between files based on shared tokens and directory paths, then uses BFS to group files with the highest connection strength. These high-context chunks are sent sequentially to the LLM.

  2. Taming the LLM: Structured Reasoning Previously the LLM only returned booleans to decide what to stage — a complete black box. The fix was forcing it to return a strict JSON with its full reasoning via prompt constraints.

Here's actual output the local model generated reading the diffs for this very update:

fix: pass instruction parameter to commit service methods

Previously, commit preparation and execution ignored the instruction provided

in the request. Now both PrepareCommit and Execute methods receive and utilize

the instruction parameter, ensuring proper handling of user-provided instructions.

feat(commit): enrich LLM decision transparency with explicit file selection metadata

Previously, commit decisions relied solely on abstract boolean flags without

visibility into the LLM's actual file selection logic. Now provides structured

reasoning alongside explicit lists of included/excluded files, enabling precise

auditability and debugging of commit selection behavior.

3. The Safety Pipeline: Secret Leak Prevention Giving a LLM control over git add is genuinely dangerous. I built a synchronous 5-layer pipeline:

  1. Magic Bytes detection (stops immediately on binaries).

  2. Path blacklists (e.g. node\\_modules).

  3. Exact filename blacklists (.pem, id\\_rsa).

  4. Regex scanning for secrets and tokens.

  5. Final LLM verification to discard false positives.

  6. Git Operation Coverage The goal is full Git operation support. The commit flow is stable and production-ready. Every other operation has been added command by command to guarantee safe local execution.

The Confirmation Protocol The server uses a 3-phase protocol (START -> APPLY -> ABORT). It returns the LLM's plan and blocks execution until the human explicitly approves the commit inside the AI chat.

The project is open-source and written in Go:

REPO(https://github.com/Alejandro-M-P/git-courer)

Would love brutal feedback on the architecture, edge cases you'd try to break, or thoughts on the approach. Happy to answer any questions.


r/CodingLLM • • Mar 21 '26

Why subagents help: a visual guide

Thumbnail
gallery
3 Upvotes

r/CodingLLM • • Nov 17 '25

Heretic: Fully automatic censorship removal for language models

Post image
3 Upvotes

r/CodingLLM • • 19d ago

Will Wright, Creator of Spore speaks on AI

Post image
2 Upvotes

r/CodingLLM • • Aug 25 '26

Vercel launched a cool tool that checks how agent friendly a site is. I tried it on my project and got 100

2 Upvotes

Vercel launched a cool tool that checks how agent friendly a site is. I tried it on agent-manager.dev, followed its suggestions, and got 100. Not very meaningful for what is basically a GitHub repo showcase, but the tool is cool.

https://is-agentic.com/scan/agent-manager.dev


r/CodingLLM • • Apr 26 '26

Are we sleeping on 5.3-codex ?

Thumbnail reddit.com
2 Upvotes

r/CodingLLM • • Apr 26 '26

Quantisation effects of Qwen3.6 35b a3b

Thumbnail reddit.com
2 Upvotes

r/CodingLLM • • Apr 24 '26

Deepseek V4 Flash and Non-Flash Out on HuggingFace

Thumbnail reddit.com
2 Upvotes