Survey data quality: How much of your survey data is actually usable

Survey data quality is getting worse. Over the last 12 months, GroupSolver removed an average of 54% of survey completes for fraud or inattention, and the rate has trended around 60% over the last three months. In a live session on September 24, 2026, GroupSolver CEO Rasto Ivanic showed real respondent data and walked through the four measures that decide how much of a dataset survives.

Short on time? Jump straight to the specific section:

• What usable survey data means
• Why survey data quality keeps getting worse
• Why a clean fraud check still leaves inattentive respondents in your data
• How to protect survey data quality: four measures that work in practice
• What a quality check looks like in a real study
• What the industry has not solved yet
• Key takeaways
• FAQ

What usable survey data means

Usable survey data comes from real people who paid attention and answered in good faith. Two separate problems break that definition:

• Fraud: bots, click farms, and people who use AI to answer questions on someone else’s behalf. Here is how to spot AI-generated survey responses .
• Inattention: real people who click through and add almost nothing to your data.

Most survey data quality processes treat the two as one problem. A respondent who answers “fine,” “good,” and “n/a” to every open-ended question passes a fraud check. They verified their identity and answered every question. They still tell you nothing.

Rasto was direct about the distinction: fraud is bad quality, and so is inattention, and both have to be removed. They just need different checks.

Why survey data quality keeps getting worse

The study that changed how GroupSolver works started with a stall. A B2B project targeting educators, higher education administrators, and professors in the US (21-minute average length, programmatic and custom recruitment) was struggling to fill. The team asked the supplier to change recruiting, and within a day a large batch of completes arrived. Rasto read the answers himself.

The waterfall tells the story. Of 3,849 respondents who entered the survey, 1,302 completed it. Traditional in-survey checks, the kind most teams rely on, removed 112. A review afterward removed another 790, which left 400 acceptable completes. In total, 69% of completes were fraudulent or inattentive, and no supplier disputed a single removal because the evidence was clear.

That result raised a bigger concern about what else was sitting in the data. GroupSolver paused platform development for about six months and pointed its whole engineering team at building quality measures into the survey.

The tracking that followed is uncomfortable. When GroupSolver started measuring, about 30% of completes were low quality. The three-month rolling average has since roughly doubled, to around 60%, and appears to be stabilizing there. It is possible that the fraud is growing, or that the checks are getting better at finding it. Either way, the rate has not come down.

Two consequences follow:

• Cleanup costs days. Survey data quality work done after collection, such as reviewing open-ended answers to find patterns, takes far longer than stopping bad respondents during the survey.
• Conclusions shift. Bad respondents do not just add noise. They answer differently from real ones, as the examples below show.

The second consequence is the expensive one. It is when bad data costs real money .

Why a clean fraud check still leaves inattentive respondents in your data

Fingerprinting tools check whether a respondent looks real. They do not check whether the respondent tried. In the Gen X and the economy study Rasto ran the weekend before the session, GroupSolver deployed two independent fingerprinting tools at the start of the survey. The tools disagreed sharply: Tool A flagged a total of 8 respondents as fake or suspicious, and Tool B flagged 97.

A closer look at survey data quality in this study showed why both matter. Among the 167 respondents GroupSolver terminated in the survey for quality, 118 were flagged by neither tool. Some of them may be real people who gave low-effort answers. And among respondents who passed GroupSolver’s checks, the tools flagged fake or suspicious profiles at about the same rate as they did among the bad ones.

In Rasto’s words, the two are orthogonal measures. Identity checks and behavior checks measure different things, so a quality process needs both.

How to protect survey data quality: four measures that work in practice

Rasto was clear that no single fix solves this. These are the four measures GroupSolver applies to protect survey data quality, and he invited other researchers to share what works for them.

1. Design the study to expose inattention

Every GroupSolver survey includes at least two attention checks, three or more open-ended questions, and at least one logic check. Simple checks like “select disagree to continue” or “what is six plus three” no longer stop AI agents. Current checks are built around known weaknesses of AI models. They are easy for an attentive person and hard for a bot or a distracted respondent. They are drawn at random from a tested pool, placed at natural breaks in the survey, and retested monthly against GroupSolver’s own bots.

Logic checks compare answers from different points in the survey. One example asks for age at the start and birth year at the end. A stronger version asks an open-ended question, then later rewords it as a negative statement and asks whether the respondent agrees. As Rasto put it, “everybody’s human until proven otherwise,” so one inconsistency proves nothing. Repeated ones do.

Study design shapes survey data quality too. The longer, denser, and more grid-filled the survey, the fewer qualified humans finish it.

For the full playbook, see the four data quality measures every survey needs .

2. Terminate in the survey, but not immediately

An attack on a survey is a volume game: once someone finds the path through, they replicate it with small variations. GroupSolver lets flagged respondents finish, then terminates them at the end. That hides the qualifying path, reveals the fraudster’s patterns, and wastes their time. Since March 2025, GroupSolver has terminated more than 25,000 respondents this way, with a 100% panel reconciliation rate. Anyone who contacts GroupSolver about a false positive is offered payment.

3. Choose suppliers by their removal rates

Pull quality termination rates for every supplier and stop working with those above a reasonable level. In GroupSolver’s supplier comparison, removal rates across exchange suppliers ranged from 21% at the best to 94% at the worst. The comparison needs repeating, because a source that performed well last month can be infiltrated this month. “We don’t blindly trust anyone ever,” Rasto said.

Supplier choice shows up directly in results. See how cheap survey panels corrupt pricing research .

4. Show clients what bad data does to a decision

Good data costs more, and clients pay for it only when they can see the difference. The clearest example from the session is a Van Westendorp pricing research study for a new product. With the 500 respondents who passed quality checks, the data pointed to an optimal price of $45. The 299 least obvious bad respondents, who give plausible open-ended answers but fail on logic, pointed to $75 on their own. All 843 bad respondents together pointed to $100.

The client would have launched at more than double the price the market would bear, which is often the difference between a successful launch and a failed one. GroupSolver now includes a data quality report as a standard deliverable, so clients can see the split in their own data. Paying more for verified quality also rewards the suppliers who deliver it, which Rasto called a circular positive economy.

What a quality check looks like in a real study

The Gen X and the economy study was a fairly generic consumer survey, fielded the weekend before the session. It shows the four measures in action, with most removals happening during the survey and fewer after it.

Of 1,024 respondents who entered, 250 dropped out and 397 were terminated or over quota. Another 167 were terminated in the survey for quality, and 41 more were removed afterward, leaving 169 completes. Of the 377 respondents who reached the end of the survey, 208 were removed. Because the in-survey checks catch the bulk, the post-collection review covers a much smaller set, which allows a more thorough look.

That review has four steps:

• In-survey behavior: copy-pasting, focus loss such as switching tabs, and suspicious clicks or mouse movements (2 removals).
• Fingerprinting: anyone flagged as fake or critical by either tool was removed; “suspicious” flags were not automatic removals (13 removals).
• QC script: 17 flags focused on open-ended answers, such as repeated answers, lazy answers, answers that echo the question, irrelevant answers, and profanity (12 removals).
• Manual check: a review of all open-ended answers, followed by updates to the script (14 removals).

The removals changed what the survey data says. Valid respondents felt worse about their personal economy in 2026 than the terminated ones, and they worried more about inflation in open-ended answers. The largest gap appeared on retirement readiness: valid respondents were far less likely to say they were on track. Rasto noted the differences here are modest because the topic is generic, and that studies with a narrow audience or specialized knowledge typically show bigger gaps. The direction of the bias also varies. Bad respondents are not consistently more positive or more conservative; sometimes their answers are simply random.

Rasto demoed these good-versus-bad comparisons live in Agatha , GroupSolver’s AI research analyst.

What the industry has not solved yet

Two gaps came up in the session, and both are open invitations.

The first is fingerprinting thresholds. Researchers own the decision about how many false positives they will accept, such as real people flagged because someone else on their IP address runs a proxy. But Rasto is not aware of any public, replicable study that quantifies false positives and false negatives for the available tools. “I’m happy with 10% of noise,” he said, “but there is no data to help me understand what level I should be setting those tools at.” An attendee suggested comparing survey results between flagged and unflagged good completes. GroupSolver has not done this yet, and its sample of flagged respondents is small.

The second is trap question supply. Giving every respondent a unique trap question would stop a bot or click farm from learning any single question. The constraint is finding 500 good ones. GroupSolver tests candidates against its own bots, and many that look strong fail.

Key takeaways

• More than half of GroupSolver’s completes are now removed: 54% on average over 12 months, around 60% over the last three.
• Survey data quality depends on catching fraud and inattention, and identity checks alone miss the second.
• Use at least two attention checks, three open-ended questions, and one logic check, and refresh the trap questions monthly.
• Terminate flagged respondents at the end of the survey to keep the path hidden.
• Compare supplier removal rates regularly. They ranged from 21% to 94% in GroupSolver’s data.
• Show clients the cost of bad data. In one pricing study it moved the optimal price from $45 to $100.

FAQ

Survey data quality now decides whether a dataset can support a real decision. GroupSolver’s numbers show a problem that has doubled since tracking began, and the fix for poor survey data quality is a set of layered checks: smarter study design, delayed terminations, supplier scrutiny, and proof of impact for clients. No one has solved it, which is why the conversation matters.

Stay curious. Ask why.