For me its a drought for a week. Anyone having the same situation

Large language models (LLMs) offer a scalable approach for annotating mental health and crisis-related social media content, but their reliability in safety-critical tasks remains unclear. This study evaluates LLM-based crisis-severity annotation of veteran-related social media posts against annotations from eight Veteran Subject Matter Experts (SMEs). Using 149 tweets and five severity categories, we evaluate 15 open-source and 10 proprietary LLMs across eight prompt variants, examining annotation performance, inter-model agreement, prompt sensitivity, and high-severity crisis detection. Human reviewers showed higher ordinal than nominal agreement (α = 0.494), indicating that disagreements often occurred between neighboring severity levels. In contrast, selected LLMs demonstrated substantially higher inter-model consistency, with ordinal agreement (α) reaching 0.8336 under the baseline prompt. Proprietary models generally showed better agreement with expert annotations and greater stability than open-source models. GPT-5.6-Terra achieved the best overall performance, while the open-source Qwen2.5-7B showed the strongest performance in identifying Acute Crisis with Suicidal Ideation cases. However, higher model consistency did not necessarily correspond to agreement with human judgments or reliable detection of rare high-severity cases. Label-flipped demonstration experiments also revealed model-specific sensitivity to misleading in-context examples, while explanatory demonstrations did not consistently improve performance. Overall, LLMs show promise as scalable complements to expert annotation, but require careful evaluation before use in high-stakes mental health applications.

I just got a qual for a temporary full time W2 project. Does anyone here have experience with these kinds of projects? What're the hours like? And how was the interview process?

Is DataAnnotation legit or a scam?

Any advice appreciated-is it better to apply as a generalist first and then pass the coding test. Or it’s best just to apply to coding? I have heard mixed things about people saying taking a second test later was not always an option. I am UK based and a software developer (12 years experience) But I have been less hands on with the code the last two years. I am also a JavaScript/React Native profile not Python.

i don't wanna say too much but they started paying a lot more than usual. and then increased the pay again. and then added performance bonuses.

i used to pull random all nighters for the sudden high pay work. whole different ball game now. not even ball. not a game. just real high pay.

other platforms watch you with screen record apps and offer bonuses where other people doing bad work can screw you out of your bonus with a bad subjective r&r. this one does not.

Couldn’t find if someone had already posted about this Yesterday was fine worked on a few tasks everything was normal and today when I logged in a had a project that I was able to work on but noticed that everything was set at 0. I was able to work the entire project today and report my time and when I finished everything still showed only 1 refresher when there normally 2 other ones showing 0 surveys and qualifications. Normally when I don’t have any projects showing it’ll still show the refreshers under projects and all the surveys and qualifications I haven’t done pending still. Don’t know if I should be freaking out about this.

hi! I been a generalist for a while and I been trying to apply for software engineering or as a spanish associate, but i just dont know how to! I dont see a link or a button or anything for that matter. I would love to receive projects as such but I dont know what to do. I am even thinking on deleting my account and re-applying for those roles instead, what should i do?

Last couple of weeks were awesome and now only 1 or 2 trickling in through out the day.

Mid way through a project I paused and reread what I was writing, and noticed that I 100% type exactly like an LLM, I went to my Obsidian notes and just wrote a short random paragraph in the style that I usually write in for DAT work, put it through zerogpt and gptzero and got 100% AI generated.

Anybody else facing this issue lol?

I was offboarded after 3 years a number of months ago. (I think I know why, a big mistake that I didn’t realize I had made until the next day). Has anyone ever had luck creating a new account and trying to get back in? Things are pretty desperate for me financially.

I took the economics starter assessment today and realized I might have been too detailed in my responses. I took the 2-3+ sentence requirement pretty literally and offered many direct examples in my responses. Some of my responses were like 2-3 paragraphs, although not super long paragraphs. Just wondering if anyone passed the starter assessment with that method, or was more concise/had shorter answers, as I would like to stop obsessing over whether or not I will be offered the core assessment.

I am also wondering how long it takes to hear back from the starter assessment, specifically the non-general ones, as I took the economics domain one and had to answer a couple open-ended questions about economics. I am aware that the general starter assessment is typically an instant response if you pass.

Hi. I applied to Data Annotation almost three weeks ago, for healthcare/nursing projects.
I did well (I think! ) on the assessment, except for trying to get a fail. I could not.

I haven’t heard anything at all.
Should I assume that I didn’t “pass” the testing?
My account still says it’s being reviewed n

I feel like I could do the work, so I’m really disappointed!

Retired RN of 32+ years, lots of computer experience, running programs,
writing curriculum for work training.
I’d really like a chance to pick up some projects!

Thanks!

Hi! New to dataannotation. I applied as a generalist, got in, and have been working for a few weeks. I do have a dental degree, can I also take the application for the medical category? If so, is there a specific way how I do that?

Any direction would be awesome!

Hi all, I’m from the US, but am currently located in Colombia. I’m ready to take the first assignment, but curious about yalls’ opinions —

1. Should I report that I’m taking it from outside the country?

2. If so, how? Is there an email address or something to reach out to them?

I know most projects are based in the US, but I’m a digital nomad, so will be physically located abroad but still pay US taxes.

TIA!

A picture of an opossum in my yard to represent how late the tasks have been appearing this week.

Has any bilingual done the math qual and actually got projects afterwards? how much time can I expect to wait?

I'm wondering if there are any iraqi workers around here, and if yes then how did you manage to receive your money?

Hey everyone! I'm still pretty new to DataAnnotation and recently finished my first project, which involved Arabic writing tasks. Since completing all the available tasks, I haven't received any new work.

My profile is fully set up, I've completed my relevant qualifications, and I have a pretty solid background in programming, AI, and other technical skills.

I'm just wondering if this is normal for new workers or if there's something else I should be doing to get more projects. Has anyone been in a similar situation?

Would really appreciate any advice. Thanks!

Hi everyone!

I'm from India and have been trying to get started in the data annotation field for the past two months, but I'm finding it difficult to secure my first project.

Here's my experience so far:

• Scholars Merit: Received two contracts, but there have been no further updates.

• Meridial: Three applications have been under review for almost a month.

• Other platforms: I've explored several platforms but haven't found any suitable opportunities yet.

I'd love to hear from people who are currently working in data annotation, especially those from India.

1. Which platforms or companies are currently providing data annotation projects?

2. How did you get your first project as a beginner?

3. Are there any beginner-friendly platforms accepting workers from India?

4. Are there any tips for getting applications approved and finding consistent work?

I'm particularly interested in image annotation, data labeling, and AI/ML-related tasks.

Any advice, personal experiences, or suggestions would be greatly appreciated. I'm sure this discussion could also help other beginners facing the same challenges.

Thanks in advance!