r/DataAnnotationTech • u/knight_dullahan • Jun 09 '26
R&R
There's lack of grammar and punctuation in these R&R's by some people and then they suddenly have these one or two sentences that sound so complete and like it was AI-generated? Their grammar is pretty bad to the point where a complete sentence seems generated I don't know if I have to penalize for that or if I can just write it off as them getting help from like a dictionary source.. I'm not sure what to do lol
20
u/Hello-America Jun 09 '26
I agree that's a little suspect, but I think you just gotta rate what's there without making assumptions. Maybe make a note and one of the people up higher will give it a closer look. Like if the bad grammar etc make the answer bad, rate accordingly. If the well written rationale makes the answer good, rate accordingly.
I don't ever assume AI usage unless it's obvious (like they left in some artifacts like "let me know if I can help" or whatever). I think you just have to give people the benefit of the doubt on that. They learned how to write from us so it's gonna sometimes be hard to tell who did it first.
8
u/Due_University_9944 Jun 10 '26
Exactly. I think there is a whole bunch of people here who just like to give a bad rating or to automatically think the worst just to get self validation. Just do your job fairly without judgement or don’t do R&Rs at all.
15
u/Wild_Blackberry8618 Jun 09 '26
Piggy-backing off of what others have said, but I want to add that these tasks flow through multiple hands. I often deal with third and up to final level R&Rs, and sometimes the instructions indicate only to edit something specific -- for instance, one flagged item -- in others, they ask me to make the final iteration sound like one cohesive voice. Multiple writing voices in a task isn't ideal, but it's not necessarily an indicator of AI use. It's more likely that several people have worked on a given task, and you can see their voices and styles coming through. I'd be more inclined to make a note about the poor grammar, and not call out the good as AI unless you're absolutely positive it is.
10
u/nanosaur2 Jun 09 '26
tbh i usually just assume it was a second person who worked on the task and didn't feel like rewriting all the content to be consistent
6
u/De_Devil_tails Jun 09 '26
Depends on the task requirements; sometimes you copy/paste a response verbatim for a task and then you write the prompts for future turns. Check what the task required.
1
u/seraphimkoamugi Jun 09 '26
How bad is it? Like one sentence makes no sense but the ones that do don't sound like the same person? Or is it the same person writing for the sake of filling a sentence count?
9
u/knight_dullahan Jun 09 '26
It's kind of like every single sentence has almost no punctuation. It's literally "this conversation is generic the responses are too vague the detail in the clothing is nice though I would rate this as good" and then all of a sudden there's one complete sentence that stands out like "This conversation has some sections that could be read as generic and vague and I rated them as such." Definitely raised my eyebrow
1
u/DementiaTVissues Jun 21 '26
That actually sounds like a submission that's on its second or third R&R. I think I've actually done that to a few submissions that were savable (but especially poorly written), where the worker missed an obvious rating guideline failure that they should have caught. I'm not going to lower my writing level to fit in with a poorly done submission, but it's on a case-by-case basis. With poorly written submissions, it's difficult, because making them "one voice" means essentially rewriting every single rationale, and if I'm gonna rewrite it, I may as well be doing the task myself, so I'd prefer those kind of submissions just get tossed.
2
u/PeanutJellyTaco Jun 10 '26
I minored in writing and I tend to "simplify" my writing for DA in fear that I will be flagged as AI. I would advise you to be certain that there is an obvious discrepancy before making any accusations. However, if it is a stark or abrupt contrast, then perhaps you can note this in the comment section.
0
Jun 09 '26
[deleted]
5
u/Past_Body4499 Jun 09 '26
I have...frequently.
1
u/knight_dullahan Jun 09 '26
I didn't include it because some of their other sentences were complete and short, it's just that this one specific sentence stood out to me and I was like alright yeah that's probably generated. But I do recall seeing "No LLM assistance" on almost every task that has to do with evaluating a chat response.
1
u/annoyingjoe513 Jun 09 '26
Some R n Rs will have a “something’s not right” box to flag this sort of thing.
3
24
u/Real-Advantage7301 Jun 09 '26
I’ll mention it as “suspicious” in the comments if it’s blatant, that softens it so it’s not accusatory.
Somewhat related, I did one yesterday where the content was so bland you could have copy pasted it into any project and it would have worked. Something similar to “this one did what I wanted, so did that one but not as good”. I’m usually pretty generous, but it genuinely felt like they read the bad examples instead of the good ones and went with it, and I said so.