Publication Date
Spring 2024
Degree Type
Master's Project
Degree Name
Master of Science in Computer Science (MSCS)
Department
Computer Science
First Advisor
Robert Chun
Second Advisor
Navrati Saxena
Third Advisor
Anuj Sharma
Keywords
Distributed Computing, Infrastructure Design, Kubernetes, MapReduce, Parallel Processing, Performance Optimization.
Abstract
The combination of MapReduce (MR) & Kubernetes (K8s) strengths is not explored, and this study leverages the synergy between the two frameworks to meet the growing demands of data-intensive applications. First, this report elaborates on the existing literature work to understand the pros and cons of using MR and K8s, in what use cases these frameworks come to use, and investigates the effectiveness of research studies that explore the combination. This study aims to research the efficacy of the fusion of MR and K8s, considering these factors - application use case, infrastructure design, resource allocation, load balancing, and hypertuning parameters that could accelerate the performance gains. Through comprehensive experimentation, this report provides insights as proof of concept serving as a starting point for architectural design development. It offers a concise understanding of what to expect when integrating MR and K8s, and how performance gains can be achieved.
Recommended Citation
Chaturvedi, Shradha, "Parallel Powerplay: Optimizing Performance with MapReduce and Kubernetes Fusion" (2024). Master's Projects. 1370.
https://scholarworks.sjsu.edu/etd_projects/1370
