Publication Date

Spring 2026

Degree Type

Master's Project

Degree Name

Master of Science in Computer Science (MSCS)

Department

Computer Science

First Advisor

Thomas Austin

Second Advisor

Faranak Abri

Third Advisor

Katerina Potika

Keywords

Abstract Syntax Trees, JSDoc, Large Language Models, Comment Generation, Code Documentation, AST Compression, Reference-Free Evaluation

Abstract

In industry comments are often skipped or poorly written in JavaScript projects, making code harder to maintain and onboard new developers. Large language models (LLMs) can help to automate comment generation by receiving the full source code for each comment that’s generated, but this can become expensive at scale. This research evaluates how using parsed and compressed data from Abstract Syntax Trees (ASTs), combined with code sampling for LLM context, compares to direct prompting that provides the full source code when generating comments. We evaluate six approaches to generated comment quality: Direct prompting with full source code, AST-Only without source code, three variants of AST+Sampling (sampling every 2, 4, or 8 lines), and an AST+Minified approach. The generated comments are evaluated using the RACE framework of Zheng et al., defining comment quality scores comprised of Readability, Accuracy, Completeness, and Efficiency. We use the reference-free prompt approach of Chen et al., with LLMs used as the judge to score each RACE criteria from 0-100. The results demonstrate that all AST+Sampling and AST+Minified approaches have better quality and efficiency than the Direct approach. The Direct approach showed to have a baseline comment quality score of 67.7% overall quality whereas AST+Sampling every 8 lines achieves the highest overall quality at 80.2%, it also saves 74.3% in tokens used and had the best processing time from 9.28s to 5.90s. The AST-Only without source code approach performed the worst and shows to be a non-viable option as it needs more context to perform well.

Download

Available for download on Saturday, May 22, 2027

Share

COinS