Publication Date
Spring 2024
Degree Type
Master's Project
Degree Name
Master of Science in Computer Science (MSCS)
Department
Computer Science
First Advisor
William Andreopoulos
Second Advisor
Navrati Saxena
Third Advisor
Thomas Austin
Keywords
generative AI, llama, resume, adapter
Abstract
The primary objective of this project is to optimize the Llama language model to generate customized resumes containing domain-specific job descriptions and maintain the linguistic capabilities of the large language model. Building upon the prior research by Sumed Kale on Job Tailored Resume content generation using GPT-2, where he employed full fine-tuning of the model and demonstrated the capability of LLMs to generate resume content, it is evident that while effective, full fine-tuning has its limitations. Primarily, it is computationally expensive, which can pose constraints, especially for large models. Additionally, during the fine-tuning process, there is a risk of losing some of the pre-trained data, impacting the model's overall performance. Furthermore, utilizing a massive model for such tasks can present resource allocation and scalability challenges. Thus, there is a need for further exploration into more efficient fine-tuning techniques and model architectures that can balance computational cost with performance gains. To achieve this in this project, we have analyzed fine-tuning techniques such as QLoRa, PEFT, and the adapter approach by limiting parameter tuning and constraints for overall domain knowledge to maintain a larger LLM's performance on linguistic tasks. Datasets were combined from several sources, including Kaggle, GitHub, student submissions, and LinkedIn, and processed to finalize our comprehensive corpus. Actions such as cleaning the resume for privacy, extracting bullet points, and utilizing tools like Scrapy on those data sets to ensure efficient and satisfactory data generation. Comparing the BLEU score and Perplexity metrics for multiple adapter models will be produced, generating the superior model to perform on resume generation for job-specific requirements.
Recommended Citation
Nidadavolu, Navaneeth Sai, "Resume Content Generation Using Llama 2 With Adapters" (2024). Master's Projects. 1364.
https://scholarworks.sjsu.edu/etd_projects/1364
