Publication Date

Spring 2024

Degree Type

Master's Project

Degree Name

Master of Science in Computer Science (MSCS)

Department

Computer Science

First Advisor

William Andreopoulos

Second Advisor

Navrati Saxena

Third Advisor

Thomas Austin

Keywords

generative AI, llama, resume, adapter

Abstract

The primary objective of this project is to optimize the Llama language model to generate customized resumes containing domain-specific job descriptions and maintain the linguistic capabilities of the large language model. Building upon the prior research by Sumed Kale on Job Tailored Resume content generation using GPT-2, where he employed full fine-tuning of the model and demonstrated the capability of LLMs to generate resume content, it is evident that while effective, full fine-tuning has its limitations. Primarily, it is computationally expensive, which can pose constraints, especially for large models. Additionally, during the fine-tuning process, there is a risk of losing some of the pre-trained data, impacting the model's overall performance. Furthermore, utilizing a massive model for such tasks can present resource allocation and scalability challenges. Thus, there is a need for further exploration into more efficient fine-tuning techniques and model architectures that can balance computational cost with performance gains. To achieve this in this project, we have analyzed fine-tuning techniques such as QLoRa, PEFT, and the adapter approach by limiting parameter tuning and constraints for overall domain knowledge to maintain a larger LLM's performance on linguistic tasks. Datasets were combined from several sources, including Kaggle, GitHub, student submissions, and LinkedIn, and processed to finalize our comprehensive corpus. Actions such as cleaning the resume for privacy, extracting bullet points, and utilizing tools like Scrapy on those data sets to ensure efficient and satisfactory data generation. Comparing the BLEU score and Perplexity metrics for multiple adapter models will be produced, generating the superior model to perform on resume generation for job-specific requirements.

Download

Available for download on Thursday, May 22, 2025

Share

COinS