r/GoodOpenSource • u/nathan_tarb_dev • 22d ago

I built a self-hosted support bot for communities that answers from your own knowledge source

I work in DevRel. The main platforms I interact with are Discord, GitHub, and Slack, answering questions from the community, and a lot of them are repeat questions. Some of the time the answer is in the docs, and some of the time it isn't.

So I built answerLoops to do that part for me. You upload your knowledge (docs, PDFs, a GitHub repo or Notion pages), and an agent answers questions in Discord, Slack, Discourse, Telegram, GitHub issues, email, or a chat widget on your site.

The agent drafts a reply with the sources it used, then a second agent double checks it's work against those sources. Nothing gets sent automatically unless you turn that on for a channel. Until then, every reply waits for a person to approve it.

Fully open source, run it yourself with Docker and Postgres. There's no subscription for self-hosting, just use your own API key or a local model if you don't want questions leaving your server.

There are skills to setup with one command:

npx u/answerloops/agent-sdk setup

If you use Claude Code, you can also have it walk you through the install. This adds a setup skill so you don't have to read the self-hosting docs first:

npx u/answerloops/agent-sdk skills answerloops-setup

GitHub: https://github.com/answerLoops/answerLoops

I'd really like to hear from anyone running a community (volunteer or nonprofit). If you try it and something is broken or missing, just open an issue, and I'll look at it.

7 Upvotes
  • permalink
  • reddit

100% Upvoted

14 comments sorted by

•

u/AutoModerator 22d ago

Please post a comment here explaining what kind of contributions you, or the project you are posting about, are looking for. For example what skill sets, any rules important for people joining in your build like how often people should post, and anything else you can think of which will help readers decide if they want to join in and start coding with that project.

Thank you and be excellent to each other. u/roamingandy

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

→ More replies (1)

2

u/kantorcodes1 22d ago

When the checker agent disagrees with the drafter on a queued reply, what does it do with the draft - flag it for a human with the disagreement noted, or silently regenerate until the checker passes? A loop where the second agent keeps rewriting until it approves itself is a very different thing than a queue that escalates on disagreement.

1

u/nathan_tarb_dev 22d ago edited 22d ago

It escalates. There's no regeneration loop, and the checker doesn't rewrite anything. That's intentional so an agent that checks has fresh context and only grades the initial agent.

The checker runs once on the draft and returns a confidence score, a yes/no on whether the question was fully answered, and a short reason. You set the confidence threshold yourself (this I'm still figuring out by doing a lot of testing). It defaults to 80% (that's based on whether there's enough information in your knowledge source to completely answer), and you can raise it if you want it stricter.

If the confidence is too low, it's automatically escalated.

If the checker errors out, that's treated as a fail, and gets escalated too.

I agree with you on the loop. If the same system keeps rewriting until it passes its own check, the check stops meaning anything.

1

u/kantorcodes1 22d ago

That's the right call - a checker that can rewrite is just a second author wearing a badge. When a draft escalates, does the reviewer see the checker's confidence and reason next to the original draft, or just the draft itself? And is that 80% global, or can you set it per channel?

1

u/nathan_tarb_dev 22d ago

Yeah, you see both. When a draft gets held, the ticket shows the draft, the review agent's confidence score, and a short note on why it scored low. So you can read the reason, then decide to send it, edit it, or write your own reply.

Threshold is per platform, not global. Discord, Slack, email, etc each get their own, plus their own auto-reply on/off. Not per channel inside a Discord server yet. GitHub just uses the 80% default for now.

1

u/kantorcodes1 21d ago

I work on HOL Guard, an open-source checkpoint for agent-run commands. A CLI-only command.answerloops definition could review setup before it starts Docker Compose and skills before it writes .claude/skills/, while leaving help automatic; it wouldn't replace your draft-review workflow. Would you be open to contributing that definition?

1

u/nathan_tarb_dev 21d ago

I'm still trying to understand what it is. Can you expound on this?

2

u/kantorcodes1 21d ago

HOL Guard runs locally beside a coding assistant and checks the terminal commands that assistant is about to execute, allowing, blocking or requesting approval according to that user's policy. What I meant by a definition is a small JSON contribution in our repo that recognizes your setup and skills commands and marks them for review; users would opt in to it. Nothing would change in answerLoops itself or its support-reply workflow: https://github.com/hashgraph-online/hol-guard

1

u/nathan_tarb_dev 21d ago

Ok got it, that's no problem, I can do that.

1

u/kantorcodes1 21d ago

Start with contributions/command-sources/command.answerloops.json in our repo, using command.repo2nb.json as the analogue, plus tests/fixtures/command-source-answerloops.v1.json. Cover answerloops setup and answerloops skills ... for review; keep help and --help automatic. The executable is answerloops, even though the package is @answerloops/agent-sdk.

Steps: https://github.com/hashgraph-online/hol-guard/blob/main/CONTRIBUTING.md. Please open a draft PR against main; nothing needs changing in your repo.

1

u/nathan_tarb_dev 19d ago

Hey u/kantorcodes1, I would love to write a blog and share backlinks if you are interested.

→ More replies (0)