Publication Date

Spring 2026

Degree Type

Master's Project

Degree Name

Master of Science in Computer Science (MSCS)

Department

Computer Science

First Advisor

Mike Wu

Second Advisor

Thomas Austin

Third Advisor

Robert Chun

Keywords

LLMs, Tutoring Systems, Java, Code Generation, AI in Education, Programming Education

Abstract

Large Language Models (LLMs) are being adopted more actively to assist in pro- gramming tasks, but much of the available AI code assistants devices are encouraging the rapid generation of solutions, as opposed to assisting students in developing an understanding. As a result, they can give the students correct code without teaching on how to judge a problem and how to build solutions or how to debug their programs themselves. This study describes JavaTutor, an LLM-based tutoring system that is used to assist learners in the learning process in a three-step process of understand- ing, generating a solution, and refining it with feedback guidance. In contrast to a general-purpose (i.e., answer-only) coding assistant, like GitHub Copilot, JavaTutor has an emphasis on pedagogical support and guided learning over answer-only support. JavaTutor tested several LLMs, including Gemini-2.0 Flash, Llama-3.2-11B-Vision, DeepSeek-R1, DeepSeek-V3, and Mistral 7B and compared their performance to the performance of GitHub Copilot with Java program problems. Both execution-based and explanation-quality metrics (ROUGE, BLEU and MoverScore) were used to assess them. Gemini-2.0 Flash had the best overall execution having Pass@1 of 0.70, 20 tasks solved and 80.93 percent unit test passing rate. It also scored highest in the quality of the explanation with a MoverScore composite of 0.5654 and a BLEU composite of 0.2621. The findings show that JavaTutor is able not only to offer a robust code-generation service but to offer a useful instructional service to help students learn programming in a lucid way.

Download

Share

COinS