Citation

Reimers JA, Turner RC, Crawford BL, Jozkowski KN, Lo WJ, Keiffer EA. Pers. Individ. Dif. 2022; 193: e111612.

Copyright

(Copyright © 2022, Elsevier Publishing)

DOI

10.1016/j.paid.2022.111612

PMID

unavailable

Abstract

The use of web-based surveys for collecting self-report data is convenient for businesses and researchers, however their use can reduce the likelihood of population representativeness and increase the risk for poor data quality. Evaluation of data quality has resulted in different proportions of participant subgroups being flagged, however these differences are not consistent across studies and survey topics. Setting aside participants identified as having low quality data (LQD) can impact sample characteristics when some subgroups are reduced at higher rates than others. In this study, the incidence of LQD is compared across demographic subgroups for four data collections measuring attitudes about social issues and consumer satisfaction. Younger participants are more frequently identified as having LQD using multiple indicators and survey topics. Conversely, participants identifying as having Black or African American race/ethnicity are consistently flagged for LQD at a higher rate for one survey topic, but not the other, indicating some LQD indicators may be oversensitive to subgroups with construct complexity. To reduce underrepresentation in samples and associated analyses when subgroups have been found to exhibit higher LQD, we recommend oversampling. When LQD for a subgroup shows evidence of being construct-dependent, we recommend LQD indicators unrelated to content response patterns.

Language: en

Keywords

Attention checks; Careless responding; Data quality; Insufficient effort responding; Person-level reliability; Survey research