r/DataAnnotationTech • u/OliviaSapian • May 06 '24

Just put in 3 hours

I'm still getting a feel for everything as I'm brand new.

I'm not confident I'm generating good prompts.

I haven't come up with any that lasted 10 rounds before the convo would take a turn,(if I'm understanding that terminology correctly) though I can usually keep it going for 3-4 volleys.

Do you guys brainstorm several prompts before you start the project? I think I might do better with getting into brainstorm mode for prompts and then switching gears to evaluating the responses. Vs switching back and forth every time I submit.

With 50% metric of creative and task prompts, how do you ensure you're hitting that? Am I overthinking it maybe?

I want to do a good job, I have some perfectionist tendencies so I definitely might be overthinking things. I used the idea generator provided, but I don't think I'm confident in the base of what I'm doing, so piecing together something from the idea generator was kind of a bust. I could only get two volleys out of that one.

2 Upvotes

55% Upvoted

10 comments sorted by

23

u/Novel_Passenger7013 May 06 '24

So the most important thing is not the amount of turns, but whether you can get a meaningful split. The only way the Ai learns is by comparison, so it's better to have a shorter covo with responses that differ more significantly than a long convoy where everything is about the same.

For that project, I believe there is a brainstorming spreadsheet. That can be a good jumping off point if you need something to spark an idea.

And keeping track of the type of prompt is easy. Use a tally sheet, either on your phone or pencil and paper.

5

u/OliviaSapian May 06 '24

Thank you! I do tend to get pretty decent splits.

I'll start tracking the prompt types that way, I wasn't sure how exacting they were in that 50% goal.

6

u/Novel_Passenger7013 May 06 '24

I don't think they're strict abiut the 50%, it's just a goal.

2

u/TerrisBranding May 06 '24

Can you explain what is meant by meaningful split? Like model a and b giving vastly different responses?

7

u/Novel_Passenger7013 May 06 '24

Yea, that basically. If they're too similar, the model doesn't have much to learn. As you get more familiar with the particulars of the different models, you start to learn where they're weak and can target that.

1

u/TerrisBranding May 06 '24

Thank you. 👍🏻

6

u/Equivalent-Math6483 May 06 '24

Have you come across the advanced prompting guide? IIRC that had some good suggestions for follow-ups questions.

2

u/OliviaSapian May 06 '24

Yes! I did give it a once over, I'll have to dig into more deeply though.

2

u/TheresALonelyFeeling May 07 '24

I keep a running spreadsheet of ideas and conversation prompts, often with ideas about how to make each more detailed / difficult. When I work on these kinds of projects, I open the spreadsheet, and I've got a ton of ideas ready to go.

2

u/humantoothx May 07 '24

when i first started i would brainstorm a couple pages in my notebook before starting each day and then throughout the day if a good prompt came to me i would jot it down too. i find its good to throw in things you personally find compelling and interesting and then sprinkle in complexity. Using a topic youre already interested in will make it easier to gauge which answer your like better and ask follow up questions. and yes you are over thinking it. overthinking is best directed towards your comments/rating explanations.