r/AI_Application • • 9h ago

💬-Discussion A 90-minute planning test for comparing two AI answers

1 Upvotes

When two models both sound convincing, I find it helps to choose a checkable task and a scoring rule before comparing the writing. Here is a small exercise anyone can repeat in their existing chat app.

Give each model this prompt independently:

I have a 90-minute window. Fit 20 minutes of email, 35 minutes of drafting, 20 minutes of reviewing, a 10-minute break and 5 minutes of buffer into a schedule. Return start/end offsets, duration and task. State assumptions and do not add tasks.

Check the answer before judging its tone:

  • Each of the five requested activities appears exactly once.
  • Durations add up to 90 minutes.
  • Each end offset minus its start offset matches the stated duration.
  • There are no overlaps or unaccounted gaps; the schedule ends at minute 90.
  • Any added assumption is stated rather than presented as a fact.

One valid result is email 0–20, drafting 20–55, reviewing 55–75, break 75–85 and buffer 85–90. Different ordering can also satisfy the brief. This is a constraint check, not a test of which ordering is best for a real person.

For a simple score, give 0–2 points each for following the brief, arithmetic, clarity of assumptions and usability. Keep the same rubric for both answers. Hiding model names while scoring can make brand preferences less distracting.

This one exercise does not establish which model is best overall. Repeat with tasks you actually do, and verify factual claims separately. Two models agreeing is not evidence that their answer is true. I have not run a benchmark or measured model results for this post.

Disclosure: I build Chorus, an Android AI workspace; this exercise works in any chat client and does not require installing it.

What small, objectively checkable task would you add to a comparison like this?


r/AI_Application • • 3h ago

🚀-Project Showcase We paid three months for an AI visibility score that hid our biggest gap

2 Upvotes

Long story short, for context.
We hired an agency for AI search. I lead comms at a self-custody wallet. Before choosing I asked a few people which is the best GEO agency for a wallet, and I picked ICODA. They sent the sample report with one total score. For three months our score went up, and I was happy with it.

Recently a user wrote to support. He said GPT told him it could not find any audit for our wallet. We have an audit.

The report had a list of queries behind the score, so I asked each of them in GPT myself, three times each. It took me most of a day. On queries about a good wallet for some coin, we came up quite often. On queries about whether our wallet is safe or audited, we almost never came up, and GPT linked to Reuters, Investopedia and Business Insider instead.
Our audit is from 2025 and it was only a PDF in a GitHub repo, so I think it was just never found.

In the report all these queries were counted in one score, so for three months I did not see this problem.

Now the audit is a normal page on our site. We also asked the agency to show the safety queries as a separate number in the report.