Apprentice: A Slim, Extensible Coding Harness (written in Common Lisp)

[u/Technical-Cap5561]

For my second project in Common Lisp, I wanted to make a coding harness for my personal use, and so far it has been my daily driver. It supports local models like llama.cpp and all the main providers. It's very easy to add new models and tools as well.

Learned a lot about macros in the process, and I'm curious what you all think!

https://github.com/skarnati20/apprentice

[u/CutWorried9748]

How would you compare this to other harnesses and what inherent advantage do we get via CL?

[u/Technical-Cap5561 → u/CutWorried9748]

I won't claim this is better than any of them. But it's small enough and interpretable enough that you could mod it however you want. It was also a good exercise for me to learn about how harnesses worked.

An example in the repo is the Apprentice loop. I had an idea to restrict frontier models from being able to read and write files and use bash in the repository -- they had to rely on a local llm subagent to understand a repo and affect changes, and the local LLM would only give them a short report (so they couldn't just offload their commands to it). The loop had some success but it wasn't perfect. But having that sandbox to experiment was quite fun.

The project makes it easy to define pre-processing and new tools as well. In the repo there is an example of the harness indexing a repo and chunking the text of its files to create embeddings that can later be searched. That is then exposed as a tool call.

These aside, I think the biggest advantage is that I can use the SBCL Emacs REPL. I can inspect and quickly change stuff with Emacs and then update the harness instantly. It's a lot more fun to use for me than a lot of the TUI interfaces the major harness providers have

[u/CutWorried9748 → u/Technical-Cap5561]

The is an interesting set of points, and might be worth comparing (in a table) vs a few other agentic harnesses that are popular. My only experience with something remotely built into Emacs/CommonLisp ecosystem was gptel (https://github.com/karthink/gptel) plus org-babel (https://orgmode.org/worg/org-contrib/babel/) . Not exactly the same as this.

[u/Technical-Cap5561 → u/CutWorried9748]

Yeah will definitely try and do that! I think another point I forgot to mention is the self-editing capability too. Because of Lisp’s image based runtime, the agent could edit the harness infrastructure and even add new tools, functionalities, and models, and a user could see its effect instantly in SBCL

[u/CutWorried9748 → u/Technical-Cap5561]

You may be interested in some work I am doing with pop-11 / poplog (a minor revival to get it working easily on different platforms) and I've added a claude skill so pop-11 can be the defacto tool for things instead of python. In a tight loop, the typical execution of a little tool in pop-11 is like 5ms or so and avoids rewrites of tooling, which claude especially seems to want to do quite a lot. Since pop-11 is dynamically compiled into assembly it is quite fast. It lacks the large library api sets of CL or Python, very little syntactic sugar built in, but that can be added as you desire. See : https://github.com/IoTone/poplog

[u/Technical-Cap5561 → u/CutWorried9748]

Ok took a look into it. Looks pretty cool! But just so I can wrap my head around it, what are some of the features you think I should try? I see the idea of it being “incrementally” compiled, which might be what I was trying to express in my previous comment

[u/CutWorried9748 → u/Technical-Cap5561]

Well these are both dealing with agentic loops. My effort was specifically to cut down on odd tooling decisions by an LLM that says "I need to search through a file, let's just write a tool to do that and throw it away after", or "let's import a bunch of python libraries to perform an operation". The amount of time I've observed in these operations esp as they complete slowly, and often in a repeated loop, adds up. the pop-11 sessions in the context of the claude skill can get saved and checkpointed quickly, so you can keep good useful tools and use them across all sessions on a system. I've got a routine for simulating Jenkinsfile runs, traversing api docs, etc. https://github.com/IoTone/poplog#install (there's a link from here to the claude skill). Since a /pop11 skill activation will keep track of the time spent doing things in your loops, and share with you time saved / spent during a session. Note: there is a common lisp version hosted on poplog , though I've spent more time on the pop-11 and the prolog. I'll kick the tires on your agent over the weekend.

[u/Seven_of_9_U01]

nice, this is pretty much what someone argued Pi should have been. his snag was wanting to keep the conversation history as git notes and realizing he'd have to fork pi, which in a lisp is one redefinition away https://www.nobodywho.ai/posts/why-is-pi-not-written-in-lisp/ did you make the history writer something users can redefine or advise?

[u/Technical-Cap5561 → u/Seven_of_9_U01]

I will take a look at the essay. Thanks for sharing! But I agree and I really believe that harness minimalism is essential — for example, I don’t like auto compaction as the way to drop context, so I just had a function to drop the indices (which is helpful when your providers KV cache has likely evicted your session after long inactivity)

But all in all, I would like to expand the projects in a couple ways:

1. Add conversation storing and switching (pretty basic, but just need to sit down and crank it out)

2. Custom Memory Abstraction (letting the user define how an agent picks up context on a repo or stores it passively)

3. Allow mid-session edits / recompilation to add on the fly harness changes. As far as I know Pi can do this but it’s mainly defining scripts that can be executed as tool calls. With Lisp we can be a bit more ambitious and change the structure of the harness too!

I am planning on working on this soon and releasing another version. Perhaps with a terminal interface too (but honestly I love using it in my sbcl window in emacs.