Publication Date
1-11-2022
Document Type
Article
Publication Title
International Journal of Qualitative Methods
Volume
21
DOI
10.1177/16094069221075860
Abstract
We outline a process for using large coder teams (10 + coders) to code large-scale qualitative data sets. The process reflects experience recruiting and managing large teams of novice and trainee coders for 18 projects in the last decade, each engaging a coding team of 12 (minimum) to 54 (maximum) coders. We identify four unique challenges to large coder teams that are not presently discussed in the methodological literature: (1) recruiting and training coders, (2) providing coder compensation and incentives, (3) maintaining data quality and ensuring coding reliability at scale, and (4) building team cohesion and morale. For each challenge, we provide associated guidance. We conclude with a discussion of advantages and disadvantages of large coder teams for qualitative research and provide notes of caution for anyone considering hiring and/or managing large coder teams for research (whether in academia, government and non-profit sectors, or industry).
Funding Number
BCS-1759972
Funding Sponsor
National Science Foundation
Keywords
collaborative research, intercoder agreement, intercoder reliability, qualitative coding, qualitative data analysis, team-based coding, text analysis
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License
Department
Anthropology
Recommended Citation
Melissa Beresford, Amber Wutich, Margaret V. du Bray, Alissa Ruth, Rhian Stotts, Cindi SturtzSreetharan, and Alexandra Brewis. "Coding Qualitative Data at Scale: Guidance for Large Coder Teams Based on 18 Studies" International Journal of Qualitative Methods (2022). https://doi.org/10.1177/16094069221075860
