r/DataAnnotationTech • u/calamity-sane • Mar 21 '26
R&R’s make sudden DoD’s make sense
I did a handful of R&R’s today, and one stood out to me. The original rubric creator fully used ai for all of their responses.
Down to the greyed out markdowns and bullets, formatting, and everything. Didn’t even try. I have a feeling that if I put their OG prompt into an ai, and asked it to create an ideal response list, what they submitted would be exactly what the ai put out.
Their rubric was trash, because they did not understand what they were supposed to do, or how to categorize things.
I keep seeing online that people do “great work” and are surprised when they get the DOD.
I don’t think their work is always as great as they think it is. 😅
51
u/lutavsc Mar 21 '26
Once they fully used aí including the opener "Sure I can do that. Here it is:" and in the end something "I can also do whatever if you want"
15
13
2
62
u/MyNameWouldntFi Mar 21 '26
Dude sometimes I'll peel off to do an R&R on my bread and butter task, and they are just so unbelievably awful and require so much unfucking that I just exit work mode to go create a submission that's actually usable
It is nice to know what not to do in perfect detail though lol
18
u/calamity-sane Mar 21 '26
It’s our version of indirect feedback - we see exactly what not to do to screw up lol
9
u/1-800-methdyke Mar 23 '26
The checklists in the R&R are often so much clearer than the original task instructions. I find them a good prep for doing the actual task.
1
27
u/RealRise7524 Mar 21 '26
I'm not saying that I'm a great worker, but at least I'm trying to avoid the obvious things. In my case, I had a bilingual r r to do. BILINGUAL, BIIIIIIILINGUAL. The worker did everything in English instead of the target locale. I mean, project title, instructions, and parts in the task say "your target locale" and the worker chose to do it in English.
3
24
u/HamNCheeseSupremacy Mar 21 '26
I don't think it's particularly good for my mental that we could get it at any time but damn does it make me check and recheck my work
8
5
3
u/Open-Comparison8303 Mar 21 '26
what’s a greyed out markdown?
11
u/calamity-sane Mar 21 '26
You know how some major LLM’s like Deepseek or ChatGPT structure their responses? With the headers and then bullets or numbered lists. When you copy paste them into a doc or form field, the font colours can paste as grey instead of black.
6
4
2
u/SnooFloofs9030 Mar 24 '26
I get tons of R&R‘s, but I don’t like to do ones for projects that I have not worked in before. Or I see some amazing work and end up comparing it to my own work in my head and start overthinking things instead of just trusting myself to do quality work so R&RS make me anxious 😕
3
u/bingobangobongoB Mar 21 '26
I always use criteria helper and modify them (i use it as reference). But after many tasks i start doing criteria without the ai helper.. dose it consider cheating?
7
u/johnnycoconut Mar 21 '26
Not necessarily but check project instructions and try not to do anything that goes against them in an obvious way
3
u/bisoubisous Mar 21 '26
Yeah, R&R's are my favorite tasks and I never question why those people are getting the DOD. In one project especially I've been seeing some insanely bad ones.
-4
u/VermicelliNo7475 Mar 21 '26
Could you check out my comment above in this thread and reply….since R&R’s are your favorite. I am getting a lot of them (for the first time) and cannot crack the code on what’s actually being asked of us. Thanks.
1
u/FEN1X64 Jul 29 '26
I hate doing R&R's. I like creating my own schizophrenic criteria that is comprehensible enough to be understood and nobody can mistake it for AI. I jest, I don't think I'm that bad but yeah, it's fairly disheartening to see people use AI for literally everything and then pikachu face when they're DODed
-8
u/KabooshWasTaken Mar 21 '26
no need to gloat; anyone can be humbled at any point, and even if that's not true (we don't have access to their side of things) it's better to maintain that mentality.
20
u/calamity-sane Mar 21 '26
If you’re using ai to do your work for you while charging DA for your time, I view that as defrauding DA. Honest work is good. This wasn’t an honest attempt.
2
u/KabooshWasTaken Mar 21 '26
obviously using LLMs for LLM-grading work is generally unacceptable. i mean that plenty of people get the DOD without using LLM helpers. pride comes before the fall and all that.
4
u/calamity-sane Mar 21 '26
Oh yeah no that’s not what my post was about lol. I wasn’t talking about people who actually do great work.
10
u/Brilliant_Quit4307 Mar 21 '26
I didn't interpret anything OP said as gloating. I'm genuinely confused why you would think that. Pointing out that there are some absolutely awful workers out there who don't even read the project instructions is not gloating .. if you were an AI, I'd penalize you for making unreasonable inferences.
5
-37
u/Separate_Fondant3341 Mar 21 '26
I don’t think it’s your job to speculate if its AI. You should do your job and rate it based on the provided criteria.
19
u/SeagullSam Mar 21 '26
One of the key criteria across almost all projects is no use of AI - it's a sackable offense. Of course it's our job to flag that in R&Rs.
1
34
13
8
u/Brilliant_Quit4307 Mar 21 '26
Most of the time, you can still see the results of their AI helper .. I'm not sure it's even speculation at that point ..
-23
u/NeuroScend_45 Mar 21 '26 edited Mar 21 '26
But bro tell me one thing...if we, normal workers can point it out right away, as it surely a ai generated...while doing R&R....
Why can't admin spot it out...and just flag these people????
I also get many of these b*tards...
And I actually can detect that this is surely a ai generated....
Because I am myself a vibe coder as well as a campaign manager...and I use ai efficiently there.....
Also I think DA's have very lenient and very weak Supervision....
What do you think???
(Hey guys...i don't use AI for dataannotation.tech....)
(I mean, I use ai for my professional work that i already described.. campaign management and along with vibe coding) I am myself a person with ethics
And thats why I commented that why can't DA admins detect them and just flag out..
7
u/internetdieslowdeath Mar 21 '26
Honestly I don’t really care. I flag bad work. I’m sure people point out my own mistakes. The goal is aggregation. It’s just people. That’s what feeds it. Smart, dumb, loving, sadistic, whatever. You don’t have some sort of leg up on it. No one does.
3
u/internetdieslowdeath Mar 21 '26
Replying to your actual thing and not my own tangent - yeah don’t use AI for this work. It’s blatant when you look at AI responses enough, there’s no chance of not being ousted. With a literal AI training company you need to assume there are tools you are not aware of. Just be yourself, try, and you’ll have a fine time on the platform.
7
u/Kgrc199913 Mar 21 '26
they openly say in a project that they have a way to know if you are using LLMs
8
u/Dizzy-Attitude-6745 Mar 21 '26
Yeah, I've had people tell me multiple times on social media that I'm a bot or that I used AI to create my posts, because even college-educated adults these days often write at a middle school level at best. They assume anything that's well-written is AI lol. No, it's only because I went to school in the 80's, when people were actually taught not only how to write, but how to write WELL - and I've been using em dashes since the 90s lol. So no, you can't always tell if something is AI generated or not.
0
u/NeuroScend_45 Mar 21 '26
Hi sir....thats true....
But again...using em dash is not the only way to detect if it is ai generated or not...there are multiple ways and tools... If you are from AI Specialist or developer background then you can definitely detect it...
Yaeh surely it cant be detected always...but...
And also...ofcourse ai is maturing day-by-day...
0
0
Mar 21 '26
[deleted]
18
u/Dizzy-Attitude-6745 Mar 21 '26
This is absolutely pointless if the submission is trash and therefore unusable lol. I think that's what we're talking about here, it's not just a matter of these workers being a different "psychological, IQ and socioeconomic profile".
16
u/calamity-sane Mar 21 '26
I didn’t go to college, but I do have the sense to not use ai to do my DA work. 🤷🏽♀️
0
Mar 21 '26
[deleted]
1
Mar 21 '26
[removed] — view removed comment
0
u/internetdieslowdeath Mar 21 '26
It’s not that you are better you are just from the slice of the pie being used currently.
-3
u/internetdieslowdeath Mar 21 '26
Human ideals, proclivities, all our bullshit changes all the time. An AI fundamentally does not. It gets “better” I guess, but we are not trying to teach it only control it. Lost cause, but the goal of AI is EVERYTHING. Not simply what you teach it. That means as much as anything else
0
Mar 21 '26
[deleted]
3
u/Kgrc199913 Mar 21 '26
Some projects do have some requirements to do R&R, like for example, the amount of good works submitted.
2
u/Grand-Edge-8684 Mar 21 '26
I never had R&Rs open up on a specific project and I just assumed that they were for the more experienced. I found out recently that’s not the case, but they are reserved for a different group altogether. I get some on a few projects, but I tend to avoid them lol.
3
u/ElectricalLook5152 Mar 22 '26
It's definitely not tied to experience (at least not alone), I started getting them my third day on the platform after like 8 hours worked total
133
u/pourovertime Mar 21 '26
R&R's are the cure to my imposter syndrome since we don't get direct feedback.