Publication Date


Degree Type

Master's Project

Degree Name

Master of Science (MS)


Computer Science


Commercial anti-virus scanners are generally signature based, that is, they scan for known patterns to determine whether a file is infected by a virus or not. To evade signature-based detection, virus writers have adopted code obfuscation techniques to create highly metamorphic computer viruses. Since metamorphic viruses change their appearance from generation to generation, signature-based scanners cannot detect all instances of such viruses. To combat metamorphic viruses, detection tools based on statistical analysis have been studied. A tool based on hidden Markov models (HMMs) was previously developed and the results are encouraging—it has been shown that metamorphic viruses created by a well-designed metamorphic engine can be detected using an HMM. In this project, we explore whether there are any exploitable weaknesses in this HMM-based detection approach. We create a highly metamorphic virus generating tool designed specifically to evade HMM-based detection. We then test our engine, showing that we can generate viral copies that cannot be detected using previously-developed HMM-based detection techniques. Finally, we consider possible defenses against our approach.