Publication Date

Fall 2025

Degree Type

Thesis

Degree Name

Master of Science (MS)

Department

Computer Engineering

Advisor

Jorjeta Jetcheva; Magdalini Eirinaki; Mahima Agumbe Suresh

Abstract

Math Word Problem (MWP) is an important building block for learning math. This type of problem is particularly useful for younger audiences because solving it involves two simultaneous skill sets: reading comprehension and mathematical reasoning. With publicly available large language models (LLMs), generating additional MWPs is readily achievable. While researchers have started using LLMs as MWP facilitators, there still exists a gap in studies about the diversity of MWPs generated by unmodified, publicly accessible LLMs. For that reason, our study focused on two goals: (1) to evaluate the diversity of MWPs generated by publicly available LLMs when provided with examples and (2) to evaluate the effectiveness of alternative MWP representations in LLM-based MWP clustering, which enabled automatic diversity evaluation. Our study adopted an LLM-first approach, which used LLM as both MWP generator and MWP evaluator. After defining diversity as concrete metrics, we applied them to compare human-authored and machine-generated problems. We proposed Abstracted Context as an alternative representation of MWP and carried out a clustering pilot study. Upon collecting results, we observed three key findings. First, generating MWP 1 problem at a time yielded the greatest diversity when compared against provided examples. Second, LLMs were fairly consistent when generating and evaluating MWPs. Last, our pilot study showed that Abstracted Context was an effective representation in LLM-based MWP clustering.

Download

Share

COinS