Publication Date
Spring 2026
Degree Type
Master's Project
Degree Name
Master of Science in Computer Science (MSCS)
Department
Computer Science
First Advisor
Navrati Saxena
Second Advisor
Thomas Austin
Third Advisor
Jelena Gligorijevic
Keywords
Log Anomaly Detection, Distribution Shift, CAShift, Mahalanobis Distance, Principal Component Analysis, Isolation Forest, BERT Embeddings
Abstract
Log-based anomaly detection is prevalent in monitoring modern distributed systems, in which faults and security problems typically appear through abnormality in the logs. Although many current approaches claim satisfactory effectiveness under static environments, their effectiveness under distribution shift remains elusive. In this study, we empirically evaluate the models on the CAShift dataset that simulates realistic distribution shifts in terms of application, software version, and underlying infrastructure. Specifically, we develop a unified evaluation pipeline that employs BERT-encoded logs to facilitate comparisons between classical statistical methods and a deep generative approach. Contrary to previous studies, we employ a fixed threshold obtained by the base distribution and perform evaluations on all attack cases to impose a more rigorous test. Our results demonstrate that while the models consistently achieve good rankings in identifying anomalies under distribution shift, their effectiveness becomes highly limited when the same threshold is used across the different distributions.
Recommended Citation
Vipparthy, Harshitha, "Evaluating Log-Based Anomaly Detection under Distribution Shift using CAShift" (2026). Master's Projects. 1766.
DOI: https://doi.org/10.31979/etd.ne2d-7xma
https://scholarworks.sjsu.edu/etd_projects/1766
