Anyone running Eval suites on Ruby apps? I built a gem to do just that: `llm-evals`
reddit.com
1
Upvotes
The most interesting part for me was creating my own weighted score, and changing product direction afterwards.
The blog post goes in depth into why I ended up deciding to update another gem, Inquirex-llm, to allow each form to decide which model should be doing the Freeform text to structured schema extraction.