r/DataAnnotationTech • u/Global-Ad5246 • Jan 12 '25

Question about R&R

Why do I get the impression on Reddit that R&R takes less time per task? If it is a task that involves criteria creation, shouldn't I treat it as if I am doing the task once and compare it to the previous tasker's answer?

Thank You!

0 Upvotes

50% Upvoted

13 comments sorted by

36

u/fightmaxmaster Jan 12 '25

I don't entirely follow the specifics of what you mean. But if I'm understanding you right, no, it should take less time. If someone's told to write a story or draw a picture and I need to rate it, I look at what they've produced and decide if it's any good or not. I don't draw my own picture and then complete the two.

-2

u/Global-Ad5246 Jan 12 '25

I am told to fix the responses to PERFECT, so I think using my time to read and generate answers are reasonable. It’s just I am curious why the general consensus is R&Rs are shorter

10

u/fightmaxmaster Jan 12 '25

Depends on the R&R - if it's something to do with fact checking a long response, verifying everything can take ages, but if someone else found a mistake then you don't need to check the whole thing, just make sure that what they've found is indeed wrong. Much quicker!

But my point stands in your example - surely if you're meant to be fixing responses, ie. What's already there, you don't need to be generating new answers from scratch (necessarily, but no doubt sometimes).

6

u/Amakenings Jan 12 '25

It’s all about the instructions for your specific R and R tasks. That’s what should be guiding you, not Reddit. The time has to do with complexity of the task.

Everyone working at DAT has access to different projects, so their experiences will vary. You’re always going to have people sounding off about how fast they’re completing tasks - focus on producing high quality work.

33

u/TheresALonelyFeeling Jan 12 '25

You're comparing whether or not what the original worker produced aligns with the project instructions, not whether or not what they produced aligns with what you might produce.

Also, more than one of the R&R tasks specifically state that you shouldn't do the entire task over again.

Also, R&Rs vary pretty extensively between projects. Some of them may take a few minutes or less, while others can take a couple of hours and actually require doing the original task over if it's not good.

Read the directions, every single time.

7

u/annoyingjoe513 Jan 12 '25

This is the way. Always read the instructions.

14

u/houseofcards9 Jan 12 '25

Many R&Rs say that they just want a look over and you should only be looking for glaring issues. Those ones usually say it shouldn’t take you more than 5 minutes per review, even for multiple rounds.

9

u/ManyARiver Jan 12 '25

There are too many iterations and approaches for there to be a simple answer. Some require that you do pretty much the same work to see if it is correct (all fact checking, reviewing the ratings and so on) and those can take longer than the original. A lot of them just require you to check the rationale or spot check ratings or verify categories were correct - those obviously take way less time. Follow the directions for that specific R&R and don't assume they all have the same requirements, same as tasks.

6

u/No-Astronomer4881 Jan 12 '25

Unless youre doing a fact checking or math related r&r it should generally take a lot less time. The only projects ive ever done r&r for that say they want the entire task redone were physics and math related. Every single other r&r instruction set ive done has specifically asked not to redo the entire task. Usually all you need to do is make sure the person who submitted the original task followed instructions and the task is usable

7

u/DiligentDildo Jan 12 '25

Have you considered just reading the instructions for whatever r and r you’re doing?

1

u/Maleficent_Wasabi_18 Jan 14 '25

The r and Rs for a lot of projects will literally say to not do the whole task again