Publication Date

Spring 2026

Degree Type

Master's Project

Degree Name

Master of Science in Computer Science (MSCS)

Department

Computer Science

First Advisor

William Andreopoulos

Second Advisor

Wendy Lee

Third Advisor

Philip Heller

Keywords

Protein Language Models, Mechanistic Interpretability, Sparse Autoen- coders, Cross-Layer Transcoders, Antimicrobial Resistance, Beta-Lactamases

Abstract

Protein language models (PLMs) are shown to be powerful predictors of protein structure and function but their internal mechanisms remain poorly understood. Recent mechanistic interpretability methods have decomposed PLM representations into interpretable features, but they have not combined methods on a single biologically meaningful task. This paper tests whether an InterPLM sparse autoencoder and ProtoMech cross-layer transcoder can discover features in ESM-2 (6 layers, 8M) that can mainly discriminate between Class A β-lactamase and Class B β-lactamase with class C and D used as harder comparisons. The main goal is to find distinct features for Class A β-lactamase that are not shared by other classes. We find that both methods find distinct features for Class A β-lactamase, but the cross-layer transcoders show that the concepts for Class A β-lactamase seems to be distributed among nodes such as in layer 4 and 6 rather than one node. We also showcase a validation framework to prevent overclaiming the role of a node, and we use it to show that several strong nodes fail in some stages of the framework meaning that they cannot be the sole node that defines Class A β-lactamase.

Download

Available for download on Saturday, May 22, 2027

Share

COinS