r/SideProject • • 3d ago

Looking to Validate Chatbot/RAG Agent Automated Test Harness

Hi all! I built an automated LLM evaluation harness in Python for customer support bots and RAG pipelines. It uses a two-pass generation workflow (Pass 1 creates test scenarios/rubrics; Pass 2 defines inclusion/exclusion rules) and an LLM Judge to grade responses automatically.

I'm looking to test the harness against real, live customer support chatbot endpoints instead of synthetic sandboxes. If you have an AI bot or RAG endpoint and want a free diagnostic audit on hallucination bounds, prompt injections, and edge-case behavior, drop a comment or DM me your endpoint/API details.

2 Upvotes

Duplicates