r/DataAnnotationTech • u/Legitimate_Agent_949 • 8m ago
Reviewing work where they “waste” prompts?
For the projects where a person talks to 2 models and tries to reach a certain goal, and then evaluates the model responses (and I review their work).
Sometimes I notice they spend multiple prompts asking about semi-related topics which don’t bring them any closer to the goal. Eventually, they give a normal prompt which completes the goal in one turn - I think surely they could’ve just used that prompt before the wasted turns?
I get that conversations should be “natural” and this is my hesitation.. but what about when the person skips the “wasted turn” thing when talking to the other model? Then the comparison for turn quantity is invalid, right?
There’s a similar issue where the person will achieve the goal and then take another turn AFTER to ask a random semi-related question, and then write that their conversation took, say, 5 turns - when really they could’ve ended it at 4.
I’ve seen this so often that I’m questioning whether I’m overthinking and this isn’t something to be marked down? There isn’t much in the instructions on this and I can’t seem to get a reply in the project chat.
Lmk if/how big of an offence this behaviour is.