Publication Date
Spring 2024
Degree Type
Master's Project
Degree Name
Master of Science in Computer Science (MSCS)
Department
Computer Science
First Advisor
Thomas Austin
Second Advisor
Philip Heller
Third Advisor
Kevin Smith
Keywords
C#, Diegetic Audio, MIDI, Music, Unity, Video Games
Abstract
To better capture the relationship between the performer of a piece of music and the music itself in a video game context, I have designed NoteBlockLib, a system by which MIDI instructions are generated and processed in real time based on the motion and collision data of in-game objects. Ultimately, the movement of instruments made using this system would be driven by the animations of a character in the game. This system would thus allow the player character to interact with the performer in the game without sacrificing the relationship between the performer’s actions and the resulting music or to interact directly with the instrument itself. To show that this approach is viable, I have built three demonstrations in the cross-platform game engine Unity. These demonstrations make use of Unity’s built-in physics libraries, its C# scripting API, and a free third-party MIDI engine, and they show that this approach is a viable means of creating and performing music within a scene.
Recommended Citation
Rotunni, Kevin, "NoteBlockLib: A Library for Physics- and Animation-Driven Virtual MIDI Instruments for Use in Video Games" (2024). Master's Projects. 1349.
https://scholarworks.sjsu.edu/etd_projects/1349
