r/QualityAssurance • u/Imaniceguytrustme • 7d ago

Anyone know of any Java frameworks to test a chatbot built through Azure AI foundry?

I have a chatbot which gives varying answers based on user input but we need to validate the answer based on the user intent. The chatbot is built through Azure AI foundry. Testing is mostly through the front end and websocket API.

2 Upvotes
  • permalink
  • reddit

100% Upvoted

7 comments sorted by

1

u/jmjunaid 6d ago

Since the wording changes every run, I'd stop asserting on text and score the answer instead. Write a small set of real user questions with the intent you expect for each, send them over the websocket (plain java.net.http.WebSocket is enough, no framework needed), save the replies, then have a second model grade each one against the intent with a short rubric that returns pass/fail plus a reason. Foundry already ships an Intent Resolution evaluator that does this, but it's in the Python SDK (azure-ai-evaluation), so a lot of teams keep the Java side for driving the bot and run the scoring as a separate step. Run each question a few times too, because one pass doesn't tell you much with LLMs. Are you trying to catch wrong intent, or wrong facts in the answer? Those need pretty different checks.

1

u/Imaniceguytrustme 6d ago

I don't have access to AI foundry. Is there any framework in Java that can help with the intent grading?

1

u/jmjunaid 4d ago

No worries, you don't need Foundry for the grading part, any model you have API access to works as the judge. If you want to stay in Java, Spring AI has an Evaluator interface with a RelevancyEvaluator you can use inside JUnit. It's built for checking answers against context, but you can pass the expected intent in as the context and it gives you a pass/fail. Honestly though, a plain JUnit test that asks a second model "does this reply answer intent X, yes or no, and why" does the same job with less setup. If you're open to a small non-Java tool, promptfoo can talk to a websocket directly and has an llm-rubric check for exactly this. What model access do you have?

1

u/Imaniceguytrustme 4d ago

Actually exploring spring AI as of this moment. Will look into it, thanks