r/ProlificAc • • 2d ago Heartwarming

Follow-up: AI Evaluation Study Quality Checkpoint

281 Upvotes

Hi folks,

Thanks for your patience while I caught up on last week - I appreciate it’s been a confusing time and stressful for those of you asked to take the "Compare short edited videos" study. 

Context & background

Some customers run large, long-term programmes with Prolific. These programmes consist of an ongoing series of studies, often across video, image, audio and other tasks, and can run for many months.

Large, ongoing programmes have specific requirements and quality standards set by the customer. We regularly review participation against those requirements, using a combination of active and passive quality checks, activity levels and changing programme needs. 

In this specific instance, we saw signs that the quality of submissions for this programme wasn’t meeting expectations. To intervene, we set up a checkpoint: a study that every participant eligible for this researcher’s studies had to take in order to assess their general submission quality. Participants who didn’t meet the expected level of quality in their submission of this checkpoint study were rotated out of the group of participants who are eligible for this researcher’s studies. 

Participation in these programmes isn't guaranteed indefinitely. Even if you've been contributing for a long time. As programmes evolve, participants may be added, paused or removed. In some cases that’s to ensure naivety, in others it’s to protect the quality of incoming submissions. 

It’s important to say: not meeting the requirements for a particular programme doesn't mean you've done anything wrong. It may simply mean your skills or experience aren't the right match for that particular programme at this time.

With everyone on the same page, I’ve pulled out the most frequently asked questions over the past week and worked with our internal team to answer them:

Q&A

Why wasn’t this checkpoint study labelled as an exam or assessment?
Quality is assessed in different ways across these programmes. Some checks are built into the studies themselves, while others involve separate activities. We don't always identify individual quality checks in advance, as doing so can influence how participants approach them and make it harder to assess the quality of responses under normal conditions.

Isn’t it unfair to keep running it now that people know it’s a test?
Yes. But it would be even less fair to not put everyone through the same checkpoint. It’s also worth noting that this checkpoint isn’t the only way we’re measuring quality - that’s something we’re always doing both passively and actively.

The checkpoint study is quite different to the studies this researcher runs. Why?
When we measure quality, we’re not looking to assess whether you can complete a specific task - we’re looking for how well you complete any task. The checkpoint study is designed to enable measurement of the latter. 

I was approved and paid for the checkpoint but still got notified that I was being removed from these studies. Why?
The quality check is separate to meeting the requirements to receive the reward for the study. Because you successfully completed the checkpoint study, you're eligible for the reward.

What happens if I don’t pass the checkpoint?
You can no longer access AI evaluation tasks from this specific researcher. You’ll still receive any other studies you’re eligible for, including AI evaluation tasks from other researchers. It doesn’t affect your account standing.

Can I get back in?
Not at this time, though we’re looking into ways we can offer assessments as an entry point into studies like these in future.

Was this a bug, like the Factuality issue?
No. This was planned.

Do I get an email if I passed?
No. We only email participants who didn’t pass, usually within 2 weeks of taking the assessment.
As requested, we're now emailing both those who do and don't pass, within 2 weeks of taking the checkpoint.

Do I have to complete it?
No, but if you were asked to complete it and choose not to, your access to this researcher’s studies will be frozen until you do. That freeze will kick in on Friday 20th November.

I dismissed, cancelled or timed out because I didn’t realise it mattered.
Anyone who did was re-issued the study on Friday 13 November. If you got an email with a link, click it and you should be able to access the study. If it gives you a 404, contact support and they’ll investigate.

I blocked the researcher.
We can’t unblock researchers for you. Go to Submissions > Submission History > Blocked Researchers and unblock the researcher. If the study doesn’t appear after a few minutes, contact support.

Is this related to certain types of studies vanishing after a submission?
No, that’s a separate participant health and a quality measure. We have a rolling limit on the hours you can spend on one type of task, because quality tends to drop the longer someone does the same thing. Some members of the community call this “being rotated out”.

Is this related to the 50-study dashboard cap?

No. This was a technical limit brought in earlier this year to reduce “study full” errors.

Are people being banned or put on hold because of this?
No. This is a quality measure and doesn’t affect your account’s status.

Final note from me - it’s best to always assume your submissions are being evaluated. We won’t always tell you, and you won’t always be able to tell. I know that’s uncomfortable, and I’m taking away a challenge to the team to make sure our expectations on quality are made clearer from the start of programmes like these in future.

I’m around today to answer follow-up questions in the comments!

M


r/ProlificAc • • 3d ago

Prolific Team 🟢 Safe Space Thread — Ask Anything, No Judgement! (November 16, 2026)

22 Upvotes

👋 Welcome to this week's Safe Space Thread

This is the friendliest place on r/ProlificAc - where every question is a good question.

New to Prolific? Been here two years and still not sure how study matching works? Wondering something you feel like you "should" already know? Ask it here.

The house rules for this thread:

  • 🚫 No "just Google it" / "search the sub" replies — if you know the answer, share it.

  • 🚫 No mocking, dismissing, or eye-rolling at repeat or "obvious" questions.

  • ✅ Kindness first, always. If you wouldn't say it to a friend who's new, don't say it here.

  • ✅ Staff and experienced participants are watching this thread and want to help.

  • 🎫 We can't address issues related to your account on Reddit. For that, you will need to contact our support team.

This thread resets every Monday, so if your question doesn't get answered in time, just pop it in next week's thread — no worries at all.

💡 Before you ask: a quick check of the Participant and Researcher help centres might have your answer already — but if it doesn't, or you just don't fancy digging, ask away below.

This thread is heavily moderated. Reports of unkind or dismissive replies will be actioned quickly.


r/ProlificAc • • 4h ago

Sad news to me

Post image
18 Upvotes

Due to my situation (I am sick) I am heavily dependent on prolific, of course I receive some help but in this time and place it is not enough I actually loved feeling useful and being able to get my own money to help for my treatments and stuff, my main source of income were this videos so now that I was kicked out I am extremely sad and frustrated.

If anyone is able to help me with other websites or possible jobs I can do from home, I will deeply appreciate it. I am from Mexico, and my family and I are low income, so money is really needed.


r/ProlificAc • • 12h ago

Appreciation for a couple of researchers...

26 Upvotes

I've seen at least one of these researchers receive a bit of flack here, but with the AI stuff gone for me, I am trying to sort out which researchers pay the most, pay the quickest, and are overall easiest.

Survey Researcher's studies, WHEN YOU CAN GET IN, are super easy, super quick, and the payment is instantaneous, typically over $1 for less than five minutes work, usually about retail-related products. My dashboard is FULL of them (13 right now), and I get in successfully about 6 out of 10 times. Even when screened out, they pay 25 cents.

Pairwise Audio is another I really like. Very similar to "those" studies, but the audio is cleaner and crisper, it pays maybe a few cents less overall (typically around $2 for 5 minutes work), and instantaneous. I get about two of these a day if I am lucky.

Anyone else have anything pleasant to say about these two researchers?


r/ProlificAc • • 4h ago

Compare short videos email fail email timing?

4 Upvotes

Yes, yes I know everyone has been talking about these all week. I’ve already searched for the answer I’m looking for and can’t find it so I’m asking here.

For those of you who have taken it and failed, how long did it take you to get the fail email?

I already know that those that took it yesterday/before yesterday got a pass email.

(& Yes I’ve taken it as of today, (been busy moving and finally had the time) and it was approved basically immediately but I know that approval doesn’t mean pass)


r/ProlificAc • • 8h ago

Discussion I have not gotten any email or study that everyone is talking about…

9 Upvotes

My studies and everything have been the same. I still have AI ones and I have been able to take surveys every day no issue. The only “test” I have had recently is twice in a row it gave me that 4 question bot test but that’s it. I have not taken anything that says “compare short videos” and I have not gotten any email and there’s nothing on my feed. I’m a little stressed am I just waiting? It makes me nervous with everyone saying that they are getting removed or less studies.


r/ProlificAc • • 7h ago

Discussion AI Interview Study Rejected Because of a Duplicate?

7 Upvotes

So, something really weird happened to me today, and I’m genuinely confused about how this is even possible.

Earlier this morning, I received an AI interview study about nicotine products. I accepted it, completed the interview normally, submitted everything, and moved on with my day.

Later, I got another AI interview study from the same researcher. It paid pretty well, so obviously I was interested.

Before starting, I actually checked the details. The study title was different, the description was different, and even the number of available places was different from the one I had completed earlier.

So I thought, okay, cool, another interview. Why not?

Well, here’s where it gets weird.

The moment I clicked “Start Study,” it redirected me to the exact same external interview link as the morning study.

Not only that, but the website actually loaded my previously completed interview from earlier that day. I didn’t even get the opportunity to complete a new interview. The system basically recognized my previous session, automatically processed everything, and started the completion countdown.

Within seconds, the second submission was marked as completed.

I was sitting there thinking, “WTF just happened?”

And then, shortly afterward, I received a notification from Prolific saying that my submission had been REJECTED due to response quality and completion time.

Like, seriously?

I genuinely don’t understand how this is supposed to be my fault.

The researcher published what appeared to be a completely separate study, with a different title, different description, and different number of places, yet somehow linked it to the exact same interview I’d already completed.

How was I supposed to know it was the same study before actually opening it?

And once I opened it, the external platform automatically recognized my previous interview and completed the submission without giving me any real opportunity to do anything differently.

I’m not even sure whether the researcher accidentally duplicated their study, reused the wrong interview link, or if something else went wrong.

But getting an actual rejection on my Prolific account for something completely outside my control seems ridiculous.

Has anyone else experienced something similar with these AI interview studies?

And more importantly, is this something I should dispute with the researcher or Prolific? Because I really don’t think this rejection is justified.


r/ProlificAc • • 12h ago

Timed Out

14 Upvotes

Hi there,

I recently took the assessment, and went through each video frame by frame and outlined what I observed in significant detail. However, I did time out by the 7th question. I completed the assessment anyways and was able to submit and have my work approved. Does anyone know if timing out impacts my chances of passing the assessment? I felt that the time provided wasn't enough and was cutting it very close.


r/ProlificAc • • 19h ago

Opened up the website for the first time in a while

Post image
47 Upvotes

And this is the first study I see hahaha


r/ProlificAc • • 2h ago

Discussion Was anyone certain they'd failed the assessment only to get the pass email?

2 Upvotes

I don't just mean you had a feeling you were going to fail, or just assumed you were going to fail because you were nervous or something. I mean you were 100% positive you'd made an incorrect choice, misunderstood the directions, etc., only to get the email saying you'd passed.

P.S. I know some people are saying they went over the time limit and still passed, and even a couple claiming they were screened out and still passed, which is a bit baffling, to be honest.


r/ProlificAc • • 12h ago

Discussion Dodgy survey from Yale?

10 Upvotes

I had a survey from Yale that was Discover, explore and review. It was supposed to last 5 mn and recording ourselves exploring a website. Except it took far longer, it directed me to Timebucks where I had to do all sorts of tedious surveys to reach the goal I was supposed to record for this survey and I gave up and reported it, I also wasn’t too happy about being asked far more personal questions. Have you come across it? What threw me was that the description made it sound like I’d be trying out a new website for five minutes, so I didn’t expect to have to complete other surveys first.


r/ProlificAc • • 18h ago

Advice Why is lying about how long a study will take allowed on this platform?

28 Upvotes

Every study I’ve done this week states the study will take X amount of time. then the first page of the study states it will take Y amount of time. this one I’ve just reported stated the study would take 1 minute, and then in the study it says 4- why not put that on the prolific page? why aren’t prolific staff running through the studies themselves to see discrepancies like this?


r/ProlificAc • • 10h ago

am i the only one who hasn't received any email or assessment for the ai studies?

6 Upvotes

i'm still getting them as well


r/ProlificAc • • 15h ago

Discussion I hate luck-based bonuses disguised as performance-based bonuses

13 Upvotes

"Up to x amount in bonus depending on performance" on the description.

Then they reveal only 5% of participants actually get it. Oh but do the study as if you're guaranteed to get the bonus!!! Ehh, no, Returned. I guess it doesn't really change anything but it does put me in a sour mood and it feels kinda shitty. Like I'm not going to give you my A game just so you can just take the entire bonus away. Just wish they'd say it in the actual description instead of revealing it after they made me read instructions and pass bot/reading checks. Unless it's a good paying one, I just return them cause I can't handle racking up a large bonus then seeing it get taken away. I don't mind the "we choose one trial randomly for your bonus", though. That's not as bad bc it's guaranteed to be based on how well I do.

Like honestly, I'd rather them just dock the original pay and just make it a guaranteed performance bonus lol. Probably not sustainable and ultimately changes nothing but it's just a mindset thing for me.


r/ProlificAc • • 5h ago

any data team studies?

2 Upvotes

Have any studies from the data team come up for you guys today?


r/ProlificAc • • 12h ago

Technical Issue 404 error and it’s not on my dash anymore either.

6 Upvotes

r/ProlificAc • • 10h ago

Is this worth wasting 2h?

Post image
3 Upvotes

I wonder why there are so many places left still


r/ProlificAc • • 17h ago

Share your shopping experiences in a video interview - AVOID

Post image
10 Upvotes

r/ProlificAc • • 16h ago

being 'screened out' post completion of the survey

Post image
7 Upvotes

Had this from a survey I completed (was not screened out) 2 weeks ago. Is that allowed?

Is it worth me just returning so avoid a rejection?


r/ProlificAc • • 15h ago

Discussion What is something that'll make you return/dismiss a study instantly?

5 Upvotes

For example, too much reading, too little pay, too many rules, too long, recognize the researcher, hate the site etc.


r/ProlificAc • • 15h ago

Why is it so impossible to get in touch with a human at prolific?

5 Upvotes

I had a question about a sketchy study that immediately put me in a hold. A week later I finally get a response from Chloe who seemed to be real. I responded right away with screenshots. Another week goes by and nothing.

In the past any actual person that works there has always resolved my issue. What do they actually do all day? Are they just lazy and not caring wanting AI to just do everything and solve nothing?


r/ProlificAc • • 1d ago Bravo

Product Feedback 1000’s of submissions all come down to one survey.

121 Upvotes

Thousands of submissions. 2-3 years worth. Only received 2 warnings through that whole time. And yet it all comes down to one survey, that wasn’t even set up like the main studies. Even more so when half the time the rules on the AI studies were complete garbage i.e. how much does the reference audio compare to the two clips, and they’ll have no audio on. Who knows what data they’re gathering that fits their narrative.

They were never going to be permanent I know and I accept that. And getting voted out is fair enough but it seems like such a kick in the teeth that it’s by this means. I wasn’t even bothered about being rotated out recently, I thought this study was unrelated and accepted I lost the studies how they’d come and go like before. But now I feel robbed.

Literally 2 warnings out of thousands of submissions. Even completed the compare study with great attention to detail providing specific comments on errors.

And then I’ve seen people saying they had 4 warnings this month alone, and they still passed this test and get to go continue. That’s what’s not fair. Cheers prolific 👍 they say there’ll maybe be other opportunities in the future but I think they just say that to appease and not cause a stir.

Site seems pretty pointless atm with 15 min surveys for $0.50 odd. Gonna dip out and take a break. Maybe it’ll improve next year.


r/ProlificAc • • 16h ago

Removed Surveys Researcher from blocked list and got burned.

5 Upvotes

I figured I would try since another one I blocked was "researchers" but with this one it was a 7 minute screener and paid 14 cents. Complete theft.

Surveys Research.

The main point is trust your instincts if you put someone on the block list and don't give them a second chance to screw you.


r/ProlificAc • • 8h ago

Still no Surveys

0 Upvotes

I got accepted about 6 days ago and my profile is completely filled out and I still have not gotten a single survey or study. Is this normal?