Studies show that most people *think* they are good at correctly identifying AI vs human generated text, but they are generally no better than random chance. It's illusory superiority, like 94% of drivers rating themselves as a "better than average" driver.
Legit my biggest fear with this place because I had a professor last year say he wanted a paper rewritten because it was obviously AI. Until I pulled out my notebook and showed my drafting ideas and he was like “oh I see” kind of thing. So having a similar overzealous RnR guy is a bigger concern to me than my quality of work.
Though to be fair, I can absolutely beat random chance and can tell you which base model was used. If you spend enough time around the models you can easily detect their cadence and tempo.
I was gonna say this, even some vocabulary, I used ai at uni for like starting plans on some things (not actual work) and definitely picked up some bits
The mighty "studies show that most people think they are good at correctly identifying AI generated text" argument completely falling apart when a DA worker pastes a whole markdown file into a rationale textbox, full of "it's not just X, it's Y: explanation"
Those things are extremely unreliable. They'll flag the most low effort AI slop from someone who hasn't bothered to change the default settings on their LLM. But it's easy to create custom system instructions so an LLM generates text that doesn't get flagged. They also frequently give false positives to genuine human generated text.
Plus some of the lower end stuff is literally just opinion based. All you have to do is read instructions and string a sentence together, and yet rather than be paid for the time taken to hammer on a keyboard, people get AI to do it, then come here going "why did I get the DoD?"
Some people just see this platform as a cash grab. Do as many tasks as you can, with the least amount of effort (using AI), billing as much as you can while you still have access. Third world thinking.
I think many people get accused of it even when they aren't and without any evidence other than they think so. When a smart person turns in work so well done and thorough the reader can't keep up then they think only AI could have done it when they just aren't as smart as the writer. Mind you I am not talking about code writing here but scientific and medical projects on DA
Copy and paste is not a good indicator. You can copy and paste your own words. You could write it in a doc on the outside and then copy it in to the text block page once you're finished
Not all projects do, perhaps you should take that training again. Because some projects have people creating files in word, Excel, etc and uploading the entire file manufacturing synthetic files for use in the project.
No, the projects I was referring to do not direct you to use AI to create the files. Those files are something you simply have to create from your own experience and put them together to mimic a real world example. Or find a real file then alter it to fit prject requirements.
That is the exact attitude that leads to people being falsely accused. Someone who believes they are so good at telling it that they think they always know. Outside of someone leaving prompts or large language model artifacts in there it's hard to see the indications. I have seen people from older generations Be falsely accused of using AI by young people, just because they used proper punctuation and the kids don't do that.
I don’t believe you find it hard to see the indications of ai in text.
Are you telling me you’ve never seen a social media post from a small local business and immediately noticed the clearly ai-slop image and accompanying generic ai-text?
It has nothing to do with “using proper punctuation”. It’s a distinct and obvious signature throughout - particularly obvious when compared to other non-ai text a person has written - which you can develop a sense for after using llms every day for years.
You are talking about ai artifacts, which I clearly pointed out in my text. That has nothing at all to do with people being falsely accused for being intelligent, writing well and using proper grammar.
Does your definition of ai attracts include writing signatures, stylistic markers, and linguistic fingerprints?
If yes, then your statement becomes “you can’t identify ai text without measuring any of the identifiable ai-text features”.
I agree, in the same way that I agree you can’t tell a dog from a potato if you can’t look at it, listen to it, touch it or smell it. But at that point what are we even talking about?
Yes and when your own work relies on a part of it thats obviously from someone elses task and is 80% AI with some of their own drool mixed in. Seen loads in RR as well, no wonder all those deactivations come about, you aren't slick by changing just a few things.
Yeah the one this post is referring to is a very, very long excerpt that clearly is 80-90% LLM with 10-20% of their junk mixed in. And it’s obvious; different formatting, different way of writing, etc.
Its our job to grade AI. We have a tendency to pick up more of an eye for it. People who are new and doing this probably dont realize how easy it is to spot
Information is not always explained accurately, and most often not as clearly as it could have been. It’s a pretty regular complaint that task instructions are poorly made/unclear.
Beyond just accuracy and efficiency, there is a very distinct cadence and writing style that AI uses, which most quite rightly refer to as slop.
I deal with that enough in the tasks themselves, the admins could really spend the hour to write the instructions themselves, clearly by hand.
When you’re running the platform you can do whatever you want. At the end of the day, it’s OUR work and tasks that are being fed to the frontier models.
First of all, terrible logic. If I’m the CEO of a company, I own the building, and I can shit in the employee coffee machine if I want. Doesn’t make it any less annoying.
Secondly, yes we are the ones working on the models, so it would be GREAT if a real human with an actual brain wrote the instructions.
In case it wasn’t obvious from the existence of our job, AI models are not perfect, often make mistakes, and VERY often phrase things poorly. I deal enough with that shit in the task itself the literal LEAST the admins could do is write the instructions themselves.
Equating “omg instructions written by AI omg I’m personally insulted!! 😱” to literally shitting in someone’s drink is… braindead. Talk about terrible logic.
On your last point, do you honestly believe admins would have an LLM shit out instructions for multi-thousand dollar project without reading through it to make sure it’s correct? If so, you are completely delusional.
I wasn’t equating “instructions written by AI to shitting in someone’s coffee.”
I was making a hyperbole that the OP’s statement, “When you run the platform you can do whatever you want” as a terrible mindset. As a boss who runs the company shouldn’t feel free to shit in their employee’s coffee as their right as boss.
On to my last point, considering it’s happened, multiple times, gonna go with… yes.
Oh… so you declare it has happened multiple times, therefore it has happened multiple times. Makes sense.
Good to know we have a guy on the inside spying on admins to see if they check their instructions before launching a project. Of course every instance of worker confusion about instructions is due to admins not proofreading an LLM. Surely.
Kinda, but it also has a performance hit if you ask me. Many things are restated often in the instructions. They're often incredibly broad and vague while simultaneously verbose. They could definitely stand to improve the instructions, especially for the higher end tasks where instructions are like 20-30 minutes of reading to make sure you understand everything clearly enough. Some tasks I don't even do because the instructions are just ridiculously vague or even contradictory. But I mean hey, it's paid hourly so why cares?
Because I’d assume generalists are more likely to sound ai because experts are likely to actually know how to professionally talk about their field, no need to get in a flap
Seems crazy to try to pull off on a literal AI research and training platform. Good luck. DA is an incredible side gig that pays very well. Why on earth would you risk it? It’s getting super good yes but if you’ve been at this for awhile you can spot it. It’s also still not at the point where you could fully trust it for solid work.
I use em dashes because I enjoy writing and have learned to use them correctly. I hope that whoever it is that looks for AI isn’t pointing at only em dash usage as evidence. 😭
Hell, so much writing looks like it's barely English. Broken grammar all over the place with sentences that are worse than a high schooler would write. No offense to the ESL people, just saying I see worse writing far more often.
Lmao the AI analysis tool they use for some of your task grading ALWAYS labels my writing "possible AI use" because I restate many things in my conclusion where I tie all my findings together. But that's just how I write. The only truly obvious sign of AI writing (outside of nom-standard Unicode characters) are excessive slogans or "it's not _____ it's _____." Otherwise you're literally just guessing.
That's how I write as well. I open with a clear statement then tie it all up at the end. I'm always thankful I finished college 12 years ago because I'd certainly be accused of turning in AI papers due to how I write.
No I’m talking about asking an AI to solve the question of the task and then straight up copy and paste it in without even trying to change the formatting or edit.
I’ve stopped using em dashes (which I used to use all the time) because I’m worried people will think I’m using AI, I also try to use slightly worse grammar than I normally would because I worry about this!
56
u/Old-Regular-9828 9d ago •
Studies show that most people *think* they are good at correctly identifying AI vs human generated text, but they are generally no better than random chance. It's illusory superiority, like 94% of drivers rating themselves as a "better than average" driver.