Publication Date

1-11-2022

Document Type

Article

Publication Title

International Journal of Qualitative Methods

Volume

21

DOI

10.1177/16094069221075860

Abstract

We outline a process for using large coder teams (10 + coders) to code large-scale qualitative data sets. The process reflects experience recruiting and managing large teams of novice and trainee coders for 18 projects in the last decade, each engaging a coding team of 12 (minimum) to 54 (maximum) coders. We identify four unique challenges to large coder teams that are not presently discussed in the methodological literature: (1) recruiting and training coders, (2) providing coder compensation and incentives, (3) maintaining data quality and ensuring coding reliability at scale, and (4) building team cohesion and morale. For each challenge, we provide associated guidance. We conclude with a discussion of advantages and disadvantages of large coder teams for qualitative research and provide notes of caution for anyone considering hiring and/or managing large coder teams for research (whether in academia, government and non-profit sectors, or industry).

Funding Number

BCS-1759972

Funding Sponsor

National Science Foundation

Keywords

collaborative research, intercoder agreement, intercoder reliability, qualitative coding, qualitative data analysis, team-based coding, text analysis

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License

Department

Anthropology

Share

COinS