Master of Science (MS)
computer log mining, anamoly detection, machine learning, deep learning
Computer logs are a rich source of information that can be analyzed to detect various issues. The large volumes of logs limit the effectiveness of manual approaches to log analysis. The earliest automated log analysis tools take a rule-based approach, which can only detect known issues with existing rules. On the other hand, anomaly detection approaches can detect new or unknown issues. This is achieved by looking for unusual behavior different from the norm, often utilizing machine learning (ML) or deep learning (DL) models. In this project, we evaluated various ML and DL techniques used for log anomaly detection. We propose a hybrid neural network (NN) we call "CausalConvLSTM" for modeling log sequences, which takes advantage of both Convolutional Neural Network and Long Short-Term Memory Network's strengths. Furthermore, we evaluated and proposed a concrete strategy for retraining NN anomaly detection models to maintain a low false-positive rate in a drifting environment.
Yen, Steven, "Intelligent Log Analysis for Anomaly Detection" (2019). Master's Projects. 739.