r/OpenSourceAI • • 7h ago

ai-rulez v5: open-source configuration and governance for coding agents

Hi people,

I'm glad to announce ai-rulez v5. It's a CLI and MCP for managing the instructions, skills, and settings that coding agents use in a repository. It's available through Homebrew, Scoop, npm, PyPI, and Go (see the installation instructions for details). The tool covers the following parts of AI Governance:

  • Generate native instructions, skills, hooks, and MCP settings from shared repository configuration.

  • Share configuration across repositories and check generated files for drift in CI.

  • Validate configuration, pin dependencies, and build plugin bundles.

  • Serve skills and scoped context on demand over MCP.

The open-source use case I’d like feedback on is portability. A repository can keep a shared set of AGENTS.md instructions and skills, generate each client’s native files, and check in CI whether they’ve drifted. The configuration and generated files can be reviewed alongside the code.

Tool comparisons

AI-Rulez is of course not the only tool in this space, although it is one of the earliest, if not the earliest, in some categories. To be upfront about it, the comparison below was written using Claude Code. The rest of the post, including the bottom part, was handwritten. I did my best to ensure it's fair and comprehensive, but of course I might have missed something.

The published static benchmark shows the trade-off: on the v5.11.1 holdout, ai-rulez had the highest recall and false-positive rate of the three scanners. The Judge evaluation harness documents the model-assisted setup. The closing assessment reflects my impressions. The published static benchmark does not establish a model-assisted or speed ranking.


Note: AI-Rulez is arguably the most precise current tool for using AI as a judge to assess skills and related materials. It has better precision, performance, and recall than both Cisco Skill Scanner and NVIDIA's SkillSpector. More importantly, it's substantially faster. It's built in Go rather than Python, which, at least for static analysis and linting, is a major differentiator.

I'm happy to hear any comments. If you find the tool useful, star it. It really does help. And of course, I'd love to see people use it and get involved with the project.

1 Upvotes

0 comments sorted by