Dealing with Lazy Survey Respondents — Drop Them or Keep Them?
It might seem obvious that inattentive survey respondents should deleted from your data. Think again. Here’s why a “light touch” during data quality review is best.
Surprising Ways to Reduce Survey Costs: Give Respondents Money Before They Take Your Survey
One refrain you will see on these pages throughout 2021 is that the time of cheap and easy online surveys is probably coming to a bitter, crashing end. Response rates are plummeting, even lower than we thought possible. Cheap online research panels (which are used for the vast majority of market research) are rife with…
The Rigors of Good Research: Three Examples from Thought Leadership
One of the biggest challenges Versta Research faced during the first year of COVID-19 was a huge and puzzling spike in the amount of fraud on research panels. We saw many organizations falling victim to this fraud. There were obviously false studies with absurdly sensational headlines being published by the media and in scientific journals.…
Slow Down! Your Survey’s Too Fast.
It is possible now to get thousands of responses to a survey overnight so that you can turn around research results as quickly as your managers and clients want. Would you trust the findings? I hope not. Suppose you want a general population survey that truly represents all U.S. adults. You will need to ensure…
Forget the Math — For Good Sampling You Need Equality, Inclusion, and Representation
In basic stats class, all of us learned about the importance of random sampling. It provides the foundation for the iron-clad mathematics of estimation, statistical significance, and margins of error. But the problem in social science, market research, and opinion polling is that random sampling (sometimes referred to as probability sampling) almost never exists. So…
The Error in Your Smartphone Surveys
Any good researcher should agonize over mode effects in surveys. Mode effects are differences in statistical estimates caused by the “mode” through which respondents take a survey. If there are mode effects, then how the survey is conducted (by telephone, online, through a smartphone app, in person, or by mail) will affect the results, requiring…
This DIY Tool Promises Statistically Significant Research
Here is a sly little test I recommend giving the next researcher who offers up their help on your next quantitative survey. Ask if they have ever used a DIY survey tool, and what advice they would give you about sample size based on this selection tool you looked at from one of the largest…
Five Free Data Sources Every Market Researcher Should Know
For those who feel far removed from the federal government shutdowns, I hope Versta Research’s winter newsletter will inspire an appreciation for what our government provides that is invaluable (and irreplaceable) in market research. Most researchers have no idea how much free data there is from the U.S. Federal Statistical System. They should, because unlike…
Don’t Waste Your Money Boosting Response Rates
Extremely careful researchers work hard to make sure their surveys reach broad and representative samples of their target populations. This usually involves multiple attempts to reach out via multiple phone calls, e-mails, or letters of request. Since statistical models of inference presume 100% response rates within random samples, the goal is always to get response…
Beware Bad Sample from Crowdsourcing
Oddly enough, the researchers who most prize scientific rigor (our research colleagues in academia) are the last to have realized that online sampling can be treacherous. An article published in Wired magazine last week describes academic psychologists as “freaking out” because they just learned that a lot of their data, which they source from Amazon’s…