Document Type
Presentation
Publication Date
June 2018
Publication Title
Academy Health Annual Research Meeting
Abstract
Research Objective: Quality assessment is an important factor in making sure results garnered from systematic reviews have external validity. While established groups have made recommendations and developed tools for quality assessment, these tools are often complex and not applicable across multiple types of study designs, generating challenges for quality assessment efforts.Study Design: As part of a systematic review of interpersonal interventions associated with the Quadruple Aim (population health, cost, patient and provider experience), we reviewed and developed a novel tool to measure the quality of study design for included studies. A multidisciplinary team reviewed existing quality assessment tools and ranked them based on the following criteria: versatility of the tool to handle various study designs, practicality, and ease of use. We developed a Study Quality Assessment of Design (SQUAD) tool that combines and simplifies the rating tool developed by Cochrane for grading randomized controlled trials and the risk of bias criteria developed for Effective Practice and Organisation of Care (EPOC) reviews. After developing the SQUAD tool, we conducted four iterative pilot tests with two independent reviewers, and then used the final tool to rate 13 studies.Population Studied: The tool was tested with 9 randomized control trials and 4 observational study.Principal Findings: Iterative testing of the SQUAD tool yielded Intraclass Correlation Coefficients that improved from 0.58 to 0.95. The final tool consisted of 12 domains, each with three different ranking levels, 1 to 3, and an option to mark some domains not applicable based on the study design. Domains include: randomization, protection against selection bias, blinding, protection against contamination, baseline measurement, incomplete outcome data, inclusion of outcomes, exclusion of findings, acknowledgement of contradictions, protection against detection bias, reliable primary outcome measure(s), and other possible sources of bias. These domains address many elements key to conducting quality research such as randomization, adequate controls, using quality measurements, and performing unbiased analyses. We elected not to include a measure of the number of subjects, despite this being a factor in the external validity of the results of each study. We made this decision because we conjectured that if adequate and proper statistical testing was done to analyze the results of the study, the power of the test would already take sample size into account.Conclusions: The SQUAD (Study Quality Assessment of Design) tool is a practical and reliable tool for assessing the quality of randomized trials and observational studies when synthesizing findings for systematic reviews.Implications for Policy or Practice: Standardized practices for quality assessment are critical to the reliability of systematic reviews. This pragmatic tool can facilitate high-quality and efficient study assessments.
Recommended Citation
Aaron Tierney, Marie Haverfield, Shreyas Bharadwaj, and Donna Zulman. "Development and Validation of the Study Quality Assessment of Design (SQUAD) Tool for Systematic Reviews" Academy Health Annual Research Meeting (2018).
