Title
Toward Automatic Mammography Auditing via Universal Language Model Fine Tuning
Publication Date
1-1-2021
Document Type
Conference Proceeding
Publication Title
Proceedings - 2021 IEEE 22nd International Conference on Information Reuse and Integration for Data Science, IRI 2021
DOI
10.1109/IRI51335.2021.00035
First Page
215
Last Page
222
Abstract
Successful mammography screening reduces the average breast cancer mortality by 20% to 22%. However large-scale screening of mammography is subject to variability due to the inherent subjective nature of evaluating mammograms. The Mammography Quality Standards Act (MQSA) require radiologists to reports their outcome in an attempt to reduce the variability of mammography practice. Therefore to improve quality of mammography screening and to comply with MQSA requirements, hospitals deploy a significant amount of resources to track radiologists performance over time in terms of predicting cancer. One of the core bottle necks is the manual review of pathology reports by experts. In this paper we successfully pilot an AI tool that automates this review. We take a two step approach that leverages the Universal Language Model Finetuning (ULMFiT) deep learning framework. First, we train an ULMFiT model that classify biopsy reports into Benign vs. Malignant with 97% precision and a recall of 96%. We fine-tune a second ULMFiT based model that further classifies Benign reports into Benign vs. Benign (High Risk Lesion) with a precision of 77% and recall of 92%. In the process of deploying, the model corrected 24 previously mislabeled reports. Finally we present a pilot deployment study that demonstrates 94 % or higher agreement between human expert and corresponding AI model.
Keywords
Breast Pathology Reports, Deep Learning, Healthcare, Mammography, MQSA Data, NLP
Department
Mathematics and Statistics
Recommended Citation
T. B. Issa, D. Uminsky, A. Shaw, R. Makipour, and R. W. Filice. "Toward Automatic Mammography Auditing via Universal Language Model Fine Tuning" Proceedings - 2021 IEEE 22nd International Conference on Information Reuse and Integration for Data Science, IRI 2021 (2021): 215-222. https://doi.org/10.1109/IRI51335.2021.00035