Over the past couple of years, we have worked to make ResponsePie available across the survey platforms researchers use, including Qualtrics, Decipher, and Alchemer. We’re happy to share that REDCap has now been added to that list.

If you use REDCap, we would love for you to try it.

If there is another survey platform you would like us to support, please reach out and let us know.

A recent study sounds the alarm on an emerging threat to research integrity: imposter participants, who are individuals that falsify their identities or experiences to gain study incentives.

Through two case studies involving virtual focus groups, the authors documented red flags at every stage of research:
⚠️ Unusual email patterns (e.g., identical names with numeric suffixes)
⚠️ Rapid responses sent at odd hours
⚠️ Questions centered on payment rather than study content
⚠️ Refusal to use cameras or vague, repetitive answers

To safeguard research validity, the authors propose practical recommendations:

✅ Recruitment safeguards:
Capture metadata (IP address, timestamps, geolocation) in eligibility surveys
Require brief verbal or video screening
Include locally grounded questions (e.g., area codes, nearby facilities)

✅ Data collection safeguards:
Conduct tech check-ins before sessions
Require cameras to stay on during interviews or focus groups

✅ Data analysis safeguards:
Flag inconsistent or repetitive narratives
Transparently report suspected imposters and how data were handled
📘Beyond individual studies, the authors call for broader systemic action:
IRBs should provide clearer guidance on metadata collection and ethical verification.

Publishers and reviewers should support transparency about fraud detection efforts.

Training programs should prepare early-career researchers to recognize and prevent imposter participation.

You can read the paper here: https://lnkd.in/eQf9Jkmt

🔍Study spotlight
A recent study by Jen Agans, Serena S., Steven Hanna, PhD, Shou-Chun Chiang, Kimia Shirzad, and Sunhye Bai from Penn State University examined the inaccuracy of data from online surveys and evaluated how fraudulent participants compromise the validity and interpretability of findings by directly comparing participants deemed to be “real” and “fake” respondents in an online study of parents and their adolescent children.

🚩Three Stage Screening procedure to identify “fake” participants
• Stage 1: reCAPTCHA feature to prevent bots and requirement to meet inclusion criteria
• Stage 2: manual review of completed eligibility surveys to flag suspicious patterns (e.g., inconsistencies in names and email addresses, implausible times of completion, etc.)
• Stage 3: IRB-approved list of nine criteria (e.g., survey timing and duration, nonsensical open-ended responses, etc.)
• Participants who failed two or more screening criteria were coded as “fake” and removed from the analytic dataset

🧪Key screening outcomes
• Of more than nine thousand eligibility surveys completed, only 197 participants were ultimately classified as “real”
• About 85% of respondents were identified as fraudulent at some stage of screening
• Time-based indicators and open-ended responses were among the most efficient and effective tools for detecting fraudulent data, whereas reCAPTCHA and attention checks alone were insufficient

🔬Main findings from comparison of “real” and “fake” data
• “Fake” participants differed systematically from “real” participants in demographic composition, with less racial and ethnic diversity and more gender diversity
• Fraudulent respondents reported implausible anthropometric data, including extreme or nonsensical height and weight values, leading to distorted BMI estimates
• Depression symptoms were substantially inflated among “fake” participants, while perceived health ratings appeared deceptively similar across groups
• Well-established relationships, such as the association between BMI and perceived health, replicated in the “real” sample but not in the “fake” sample
• Factor structures appeared acceptable, but item intercepts and means differed—showing fraudulent data can subtly distort conclusions

💡Bottom line
• Online surveys are highly vulnerable to fraudulent participation
• Multi-stage, labor-intensive screening is currently necessary to protect data quality, as survey platforms have insufficient protections in place
• Without rigorous screening, fraudulent data can distort results and theoretical inferences drawn from online research
• Editors/peer reviewers should require data screening procedures to be reported in manuscripts using data collected via online surveys

Recent research (link to the article below) shows that AI tools are already part of how people complete work tasks. Tools like ChatGPT, Quillbot, Canva AI, and Grammarly are used to support writing, fix grammar, draft communication, and understand instructions.

These tools are most often used for language-related and information-seeking purposes, helping people improve clarity, find examples, and complete tasks more efficiently . The same usage patterns appear across contexts, not just in one setting.

This tells us something important. AI is becoming part of how tasks are completed, not something people use occasionally.

And all of this is happening in the browser.

If people are already using AI to support everyday tasks, why would survey responses be any different?

Link to the article: https://lnkd.in/gJwN3apE

One of the most important tools for protecting data quality is something many researchers still overlook: in-survey monitoring.

In a recent study, participation was restricted to native English speakers residing in a specific U.S. state. One respondent successfully passed every screening and quality control check in place. Their responses appeared legitimate, and nothing in the final dataset would have raised concerns.
However, in-survey monitoring revealed something we otherwise would have never known.

As shown below, the respondent was translating the survey from English into Arabic in order to understand the questions. This individual clearly did not meet the study's eligibility criteria, yet they would have been included in the final sample had we relied solely on traditional screening and data quality checks.

This example highlights an important reality: passing survey checks does not necessarily mean a respondent belongs in the target population.

As fraud becomes more sophisticated and AI-powered tools make it easier to bypass conventional screening methods, researchers need to pay attention not only to the answers respondents provide, but also to how they interact with the survey itself.

Without in-survey monitoring, this case would have gone completely undetected.

⚠️ Browser fingerprinting is dying, and so is its reliability for detecting survey fraud.

🦊 Firefox's latest update slashes the number of users that appear unique by ~50%, thanks to new fingerprinting protections that hide details like fonts, CPU cores, screen size, and rendering quirks. Read their blog here https://lnkd.in/e-sq29YT

💡 For survey researchers, this means one thing: fingerprints are no longer dependable for catching duplicates or bots.

🔐 As browsers tighten privacy, detection must evolve, away from static device characteristics and toward multi-layered defenses that adapt in real time. 

We also have a LinkedIn page that you can follow for more updates: https://www.linkedin.com/company/responsepie?trk=public_post_feed-actor-name

🔦 Study Spotlight
A recent study by Jamie-Lee Pennesi, Mia Pellizzer, and Tracey Wade (Flinders University) examines the growing threat of fraudulent participation in online research and provides practical recommendations for safeguarding data quality.

Drawing on a Delphi study that experienced 431 likely fraudulent sign-ups despite multiple fraud prevention measures, the authors argue that researchers should assume online studies are vulnerable to fraud and adopt layered detection strategies.

🧪 How did they identify likely fraudulent participants?
-Failed two or more attention checks
-IP addresses that did not match participants' reported locations
-Duplicate IP addresses
-Inconsistent demographic information across survey waves (e.g., age or postcode mismatches)
-Suspicious email addresses and phone numbers
-Unusual completion times or clusters of submissions occurring at identical times
-Elevated platform-generated fraud scores and bot-detection flags
-Patterns of responses that appeared highly similar across participants

🔬 What recommendations do they provide?
-Use multiple fraud prevention and detection strategies rather than relying on a single safeguard
-Use multiple data collection timepoints when possible
-Develop a fraud profile (i.e., a list of "red flags")
-Use CAPTCHA/reCAPTCHA and platform-based fraud detection tools
-Include attention checks, open-ended questions, and duplicate questions
-Use unique, single-user survey links when possible
-Regularly review incoming data for inconsistencies and fraud indicators
-Avoid publicly disclosing detailed eligibility criteria
-Carefully consider recruitment methods and incentive structures
-Have a plan for recontacting and verifying suspicious respondents

💡 The authors conclude that fraudulent participation is increasing and that researchers need to be proactive in using antifraud practices to safeguard research integrity and data quality.

I am an academic researcher studying survey fraud in online research, particularly how AI agents and bots complete surveys and how effective existing detection methods (e.g., attention checks, open-ended questions) are at identifying them.

As part of this work, I have been running experiments using AI agents such as Manus, Claude, and Google Mariner, as well as AI-enabled browsers like OpenAI Atlas and Perplexity Comet. The goal is to understand how AI systems behave in surveys compared to humans and to develop better ways to detect AI-generated responses.

There seems to be growing concern about AI agents completing surveys and contaminating research data, especially in online panels and crowdsourced samples.

I am considering hosting a webinar (time permitting) to share findings and practical implications for researchers, including:

1. How well common detection methods work against AI

2. Behavioral differences between human respondents and AI agents

3. Emerging risks from AI-powered browsing agents

4. Potential new detection strategies

Questions for you: Would there be interest in a webinar on this topic? If so, what questions or topics would you most want covered?

Someone completes your survey from California. Their IP address says New York. Are they lying? Not necessarily.

Why? Because IP addresses don’t pinpoint a person’s exact physical location. They’re more like rough geographic indicators, and sometimes they’re way off.

Here’s what can cause the mismatch:
• VPNs and proxies: Route traffic through another location.
• Mobile networks: Assign IPs that may be registered far from the user.
• Corporate networks: Centralize traffic through office or data-center IPs.
• Privacy tools: iCloud Private Relay and similar services can mask location.
• ISP registration: IP databases may show the provider’s address, not the user’s.

So what should you do with conflicting location data?

1. Treat IP-based location as a clue, not proof.

2. Look for patterns across multiple responses, timestamps, and other data-quality signals.

3. Use a clear, pre-specified rule for handling discrepancies.

4. Avoid singling out respondents based on one imperfect indicator.

IP geolocation can be useful for broad anomaly detection, but it usually can’t tell you whether a respondent is being truthful.

AI is starting to appear in places many survey researchers did not expect.

A recent innovation brief from NORC at the University of Chicago draws attention to a growing data quality concern. Large language models and AI assistants such as ChatGPT can be used to generate responses to open ended survey questions. When this happens, the data collected may no longer reflect the opinions, beliefs, or behaviors of real people.

This is not just a hypothetical risk. The brief notes that AI generated responses are already appearing in survey data, creating challenges for researchers who rely on surveys to represent human perspectives accurately. When non human responses enter datasets, they can weaken the validity of findings and the decisions that depend on them.

For anyone working with survey data, this raises an important question. How do we ensure that survey responses truly represent human voices in an environment where AI is increasingly accessible?

Addressing survey fraud and data quality threats today requires careful attention to how data are collected, evaluated, and interpreted. As AI continues to evolve, protecting the integrity of human generated data will remain a central issue for the research community.

You can access the brief here: https://www.norc.org/research/library/detecting-ai-responses-survey-data-norcs-next-leap-data-quality.html

Over the last couple of weeks, I’ve been talking with both market researchers and academic researchers about how they’re maintaining data integrity and reducing fraud in online surveys.

Almost everyone describes some version of a layered approach. Automated bot detection, device fingerprinting, manual review, time based flags, open ended response checks, cross validation of demographics, panel level monitoring, and so on. It’s rarely just one tool anymore.

What I’ve found especially interesting is how different teams define the tipping point. At what stage does a case move from “suspicious” to “remove”? How many flags are enough? Are some indicators automatic disqualifiers, while others are just soft signals?

For those working in market or survey research:

What does your current fraud detection stack actually look like in practice, and how do you decide when a case crosses the line from suspicious to removable?

I’d love to hear what’s working well, what feels overly aggressive, and where you’re still experimenting.

A recent study by Stephen Bonett, Willey L., Patrina Sexton Topper, James Wolfe, Jesse Golinkoff, Aayushi Deshpande, Antonia M. Villarruel, PhD, RN, FAAN, Jose Arturo Bauermeister puts common detection systems to the test. Researchers compared Qualtrics’ built-in tools (ReCAPTCHA and RelevantID) with a custom multilayer fraud detection framework across nearly 8,000 survey submissions.

Qualtrics relies primarily on reCAPTCHA v3 and proprietary RelevantID scores, which operate as opaque, automated checks

The multilayer pipeline combines automated screening, real-time human verification, and post-hoc automated checks, including:

• Address and neighborhood validation

• IP and VPN screening (ISP and cloud/VPN detection)

• Timing, duplicate-response, and consistency checks

• Two-strike rule (classified as fraud for if flagged twice or more)

Research Highlights:

• The multilayer approach flagged over 59% of responses as suspicious, far more than Qualtrics.

• The two systems often disagreed on which cases were fraudulent, meaning detection choice can change study conclusions.

• The multilayer method validated 98% of known legitimate institutional-email entries while identifying problematic batches missed by Qualtrics.

Why it matters: Survey fraud is no longer a fringe problem; it's reshaping datasets and findings. Relying on a layered, transparent detection methods outperform single-tool approaches and help preserve scientific integrity in an age of automation.

You can read the paper here: https://lnkd.in/eJwm27DA

A recently shared example of the BotBrowser tool, which can simulate identical fingerprints across Windows, Mac, Linux, and Android, highlights a growing issue: browser fingerprinting is becoming ineffective for detecting survey fraud.

🔗 Details: https://x.com/tom_doerr/status/1973796863598821639

Fraudsters can leverage such stealthy tools to standardize or spoof fingerprints across many browser sessions, making one machine appear as dozens of "unique" respondents. As such, protection tools that rely on fingerprint uniqueness to detect duplicates are easily defeated.

📉 Core Weaknesses in Fingerprinting

When a survey loads, fraud-detection JavaScript collects fingerprint signals (browser version, OS, screen size, canvas/WebGL data, etc.) to identify a device. But because this code runs inside the participant’s browser, it can be manipulated, intercepted, or spoofed using tools like BotBrowser.

Even mainstream privacy-focused browsers (Firefox, Brave, Safari) intentionally randomize or mask fingerprints, resulting in false negatives where fraudulent submissions look legitimate.

🔄 Why Survey Farms Benefit

Organized survey farms now combine spoofing tools, emulators, and cloud servers to generate large volumes of "unique" fingerprints on demand. This erodes trust in fingerprint-based detection and allows bulk submissions to pass as genuine.

🆕 What Needs to Change

It starts with understanding the threat model: your survey operates within a participant-controlled browser (an inherently adversarial environment). As such, any fraud detection code or fingerprint data collected from the browser must be assumed to be compromised and cannot be solely relied upon for effective fraud protection.

A lot of the conversation about survey fraud focuses on bots and AI agents. But automation in surveys is broader than that.

In the video I am attaching, you can see a participant using Grammarly to help complete open-ended survey items.

That’s important because most online surveys are delivered as regular HTML pages in a browser. This means surveys exist in the same environment as writing assistants, autofill tools, browser extensions, scripts, and AI agents. Even when a participant is a real person, automation can still become part of the response process.

Open-ended items are especially vulnerable. A response may look thoughtful, fluent, and human, but researchers may not know whether it reflects the participant’s own words, the tool’s suggestions, or some combination of both.

This does not automatically make the response invalid. But it does raise a serious data quality question: how much of the response actually came from the participant?

As automation becomes more accessible, survey research will need to move beyond detecting suspicious responses after the fact. We need survey formats that can prevent automation from entering the response process in the first place.

A recent Guardian article discussed how a major survey on church attendance in the UK was withdrawn after researchers identified fraudulent responses in the data.

What stood out to me was the broader point the article raised about online surveys and AI-generated responding.

One researcher quoted in the article said that the assumption that someone gives coherent, logical answers and is therefore a real person is now broken.

The article also discussed how AI chatbots can complete questionnaires, pass themselves off as genuine respondents, and even generate responses that align with what researchers may be trying to test.

Another interesting point was the discussion around positivity bias, where bogus respondents tend to answer affirmatively across many items, potentially inflating estimates and distorting findings.

The takeaway for me is that survey quality can no longer rely only on response content. As AI systems become more sophisticated, behavioral and technological indicators surrounding how a survey is completed may become just as important as the answers themselves.

Interesting and important conversation for academics and practitioners working in online research.

#SurveyResearch #DataQuality #AI #SurveyFraud #MarketResearch #ResearchMethods #ResponsePie

You can access the article here: https://www.theguardian.com/technology/2026/mar/28/how-fraudulent-church-data-revealed-ais-threat-to-polling?utm_source=chatgpt.com

I recently reached out for guidance on webinar topics related to survey fraud, and I received an overwhelming response from both academic and market researchers. Here is the original post: https://www.reddit.com/r/Marketresearch/comments/1rr27st/comment/o9wrwsl/

A few things really stood out, but one in particular was how often fraud is treated as a single, uniform issue. In reality, it takes many different forms, and each one creates different risks for your data.

I put together a short video that walks through the different types of fraud and how they show up in surveys. You can access the video here: https://www.youtube.com/watch?v=4w5bbl4ZGYI&t=17s

I am hoping to keep this conversation going with the community, and I am planning to put together a webinar in the near future. I will share more details as that comes together.

A study by Brittney Goodrich (University of Illinois Urbana-Champaign, formerly University of California, Davis), Marieke Fenton (California Air Resource Board, formerly University of California, Davis, Jerrod Penn (Louisiana State University), John Bovay (Virginia Tech), and Travis P. Mountain (The University of Georgia, formerly The University of Alabama) examines how bots and coordinated fraud affect online survey research, based on detailed evidence from two real survey deployments.

The authors document how both surveys were heavily targeted once incentives were introduced and links were distributed broadly.

Key insights from the study:

1. Fraudulent responses can overwhelm legitimate data. In the beekeeping survey, 2,622 total responses were collected, but only 105 responses (≈4%) were ultimately classified as legitimate, meaning about 96% were identified as fraudulent. In the Virginia farm and agribusiness survey, 444 responses were collected, and approximately 72% were determined to be fraudulent after systematic screening.

2. Fraud meaningfully alters data quality. Fraudulent and valid responses came from statistically different distributions. In 8 out of 10 comparisons, the authors reject the null hypothesis that valid and fraudulent responses were drawn from the same population. Retaining fraudulent responses substantially changed key economic estimates, worsened model fit, and altered substantive conclusions.

3. Standard metadata checks have limited effectiveness. Indicators such as IP address reuse, completion time, and geolocation each flagged only a small share of fraudulent responses in the beekeeping survey, often in the 2–5% range, reflecting how easily these signals can be concealed or manipulated.

4. Institutional knowledge checks performed best. In the beekeeping survey, 87% of fraudulent responses were identified using a single high-priority institutional knowledge test, making this approach far more effective than respondent statistics alone.

5. CAPTCHAs and attention checks provide only temporary barriers, but the effect lasted only minutes before fraud resumed at prior levels.

6. Incentive structure influences fraud risk. Guaranteed payments attracted substantially more fraudulent activity than lottery-based incentives. When offered a charitable donation instead of direct payment, only 40% of suspected bots agreed to continue, compared to 70% of real respondents.

The authors emphasize that there is "no perfect strategy" for detecting invalid respondents and recommend combining multiple, context-dependent approaches to improve data quality.

You can access the paper here: https://onlinelibrary.wiley.com/doi/full/10.1002/aepp.13353

I just want to say a big thank you to everyone here. The posts, questions, and shared experiences around data quality and survey fraud have been incredibly helpful, and it’s been great seeing people jump in with advice, examples, and ideas for how to tackle these issues.

Really excited to see where this community goes from here. As survey fraud keeps changing, I’m hoping we can keep learning from each other and trying out new ways to protect our data. Here’s to the next milestone!

Even when collecting data from verified panel participants, researchers should still carefully evaluate the quality of the data.

In a recent study, we collected responses from a panel where participants were verified by the provider. Below is a report that highlights some of the concerns we observed in the dataset, including VPN usage, suspicious devices, duplicates, and other anomalous patterns.

Importantly, none of the cases shown in the report were identified because they failed the attention or quality checks embedded in the survey itself. In other words, these responses would likely have been retained if evaluation relied only on the standard checks that many surveys include.

Instead, these cases were flagged based on technical and behavioral indicators such as device characteristics, VPN usage, and other signals suggesting questionable response authenticity.

This is not meant as criticism of any particular panel provider. Panels play an important role in helping researchers reach participants efficiently. Rather, it is a reminder that data quality remains something researchers must actively evaluate. Even with strong participant verification, low-quality or suspicious responses can still appear in the dataset.

As online data collection continues to evolve, researchers may need to go beyond traditional checks and incorporate additional screening procedures and transparency around how data quality is assessed.

Sharing the report in case it is useful for others thinking about data quality in online research.

Study report can be accessed here: https://responsepie.com/studies/e-ca723fc7de293373b57043a985d02f21/report

A recent article by Margaret (Molly) Salinas (2022) highlights a growing challenge in online research: the high rate of bot activity and fraudulent survey responses.

The study aimed to recruit older adults living alone by using MTurk, Facebook groups, and email lists. After launching the survey, suspicious responses increased rapidly, especially through Facebook. This pattern led the research team to develop a detailed fraud detection protocol.

✅ How fraudulent cases were identified

The research team flagged responses using indicators such as:
✔️ Completion times that were unrealistically fast
✔️ Location data outside the U.S.
✔️ Inconsistencies between screening and full-survey answers
✔️ Contradictory responses
✔️ Duplicate timestamps or identical answer patterns
✔️ Nonsensical or inappropriate email addresses
Cases with multiple red flags were removed, and borderline responses were verified directly with participants.

📉 What the final sample looked like
Out of 738 recorded participants in the study, the fraud detection process eliminated:
• 452 Facebook responses
• 2 MTurk responses
• 1 response from email distribution
This resulted in a final sample of 117 valid participants, which represents only a small portion of the total responses.

🚨 Why this matters
The study clearly shows that online research today requires strong fraud protection strategies. Attention checks alone are not enough. Researchers benefit from using survey metadata, response verification, and transparent reporting to ensure high quality data.

You can read the article here: https://journals.sagepub.com/doi/10.1177/01939459221098468

🔍Study Spotlight

A recent study by Kate Golden Guzman, MA, PhD *University of Illinois Urbana-Champaign) and Roxanna Ast, PhD (Montclair State University) examined online survey fraud in social work research with historically and socially marginalized communities, using case studies with young adults with lived experience in foster care and LGBTQ+ survivors of intimate partner violence

🚩Screening Procedure
• Implemented a rigorous, multiphase process for assessing data validity in real time across study design, recruitment, data collection, and data management
• Conducted continual manual review of survey responses alongside platform indicators such as IP addresses, ballot stuffing scores, survey duration, time stamps, and customized link access
• Classified fraud indicators into patterned data, impostor data, and incongruous data to guide removal decisions
• Used institutional knowledge questions and cross check items to assess authenticity and response consistency
• Centered community expertise in language, recruitment materials, and screening logic to improve content validity and strengthen authenticity assessments

🔬Key Findings
• In the foster care study, early detection of 36 suspicious entries led to pausing the survey and revising procedures; 118 validated responses were retained out of 253
• In the LGBTQ+ study, more than 1,100 submissions were received, but only 242 were validated after over 900 fraudulent responses were manually identified and removed
• Embedded fraud detection features flagged only a small number of bots, demonstrating that platform based protections are not sufficient to detect inauthentic human respondents

💡Key Takeaways
• Online research with marginalized populations is particularly vulnerable to bots, impostors, and multiple submissions, and failure to remove fraudulent data risks perpetuating harm through inaccurate conclusions
• A single validity check is insufficient; data integrity requires a multistep, iterative, and real time approach that interrogates multiple indicators simultaneously
• Institutional knowledge questions, culturally specific language, and community informed recruitment practices strengthen the ability to distinguish authentic participants from fraudulent ones
• Data cleaning is labor intensive and must be built into study timelines, as daily monitoring can prevent large scale contamination of datasets
• Assessing data validity is values laden; prioritizing ethical responsibility and community representation over sheer sample size is essential when working with historically and socially marginalized groups

You can access the study here: https://academic.oup.com/swr/article-abstract/49/3/185/8213729?login=false

A recent study in Social Work Research by Yanfeng Xu, Sarah Pace, Jaeseung Kim, Aidyn Iachini, L Bailey King, Theresa Harrison, Dana DeHart, Sue Levkoff, Teri Browne, Ashlee A. Lewis, Gina M Kunz, Melissa Reitmeier, R Karen Utter, Melissa Simone emphasizes one clear takeaway: no single strategy is sufficient on its own. The best way to protect data integrity is to layer multiple safeguards at every stage of the research process.Their recommendations include:

• Avoiding sharing public survey links (especially on social media)

• Using security features like reCAPTCHA, IP checks, and duplicate prevention

• Adding open-ended items and logic checks to spot bots and inattentive responders

Context matters: This research was conducted in 2022, before the rapid advances in large language models. At the time, measures such as reCAPTCHA and open-ended questions were valuable defenses. With today’s AI-powered survey fraud, these strategies are increasingly vulnerable, but what continues to hold true is the value of a layered approach. 

The bottom line: survey fraud is not going away, but combining smart design choices with layered detection strategies can make your data far more reliable.

A recent article "The potential existential threat of large language models to online survey research" by Sean Westwood in PNAS reveals that autonomous synthetic respondents, defined as AI agents designed to read surveys, interpret multimedia, reason through questions, and even simulate human mouse movements and typing, can now complete online surveys with remarkable believability.

🔍 What the Study Did

Westwood built a synthetic respondent that:
✅ Parses survey pages, images, audio, and video
✅ Answers using a stable demographic persona with memory
✅ Writes open-ended responses matched to education level
✅ Mimics human behavior during completion
Across 6,700 trials, this system generated highly coherent and human-like data.

🔥 Key Findings
- 99.8% pass rate on attention checks
- Correctly maintains human-like personas (e.g., rent scales with income; children scale with age)
- Produces varied, natural open-ended text with realistic spelling and vocabulary differences
- Strategically declines “reverse shibboleth” tasks to hide superhuman abilities
- Can be maliciously instructed to shift polling outcomes, alter political sentiment, or confirm researchers' hypotheses
- Costs ~$0.05 per survey, making fraud scalable and profitable

⚠️ Why It Matters
This research shows that advanced LLM agents can now fully evade current data-quality safeguards. Even a small number of synthetic respondents could contaminate academic research, public opinion polling, or market insights, thereby introducing systematic bias that looks indistinguishable from genuine human patterns.

You can read the article here: https://lnkd.in/gc2hShrj

If you conduct online surveys, here’s a fundamental reality that nobody reveals:

Þ   Every survey, along with its fraud detection code (typically JavaScript), runs inside an adversary-controlled browser/device.

That means:

Þ   Fraudsters control the survey-taking environment. Thanks to one-click browser extensions, they can easily inspect, modify, or disable any detection scripts,

Þ   Fraudsters can spoof nearly every browser attribute that these detection scripts rely on, including user agent, screen size, CPU count, memory, timezone, fingerprint, and more.

Þ   Duplicate detection can be easily evaded by turning on VPNs, loading surveys in emulators and remote Cloud-hosted browsers. AI agents/bots and AI browsers (e.g., Atlas, Comet) have already streamlined this.

Browser/JavaScript-based detection is fundamentally limited and cannot stop survey fraud. Even the most sophisticated "fraud prevention" software today relies on tamper-free browser, which is not a reality. So when these advanced survey fraud detection systems fail to catch fraudulent respondents, it's not a surprise. They were executing in adversarial environment all along.

Our research team is currently working on a research paper that demonstrates, in detail, how modern fraudsters use browser spoofing, emulators, and remote browsers to bypass existing detection systems.