Publication Date

Spring 2025

Degree Type

Master's Project

Degree Name

Master of Science in Computer Science (MSCS)

Department

Computer Science

First Advisor

Sayma Akther

Second Advisor

Robert Chun

Third Advisor

Srajan Gupta

Keywords

Human–Robot Collaboration, Smartwatch IMU, Temporal Convolutional Network, Intent Recognition, Automotive Assembly, High-Performance Computing

Abstract

In modern car manufacturing, collaborative robots (cobots) work with human operators during shared workcell interactions to maximize production speed and flexibility. Collaboration between humans and robots is safe and effective only when operator intent recognition via a single wrist-worn inertial measurement unit (IMU) is accurate and low-latency. This thesis develops an IMU-only intent recognition pipeline, and is evaluated on three datasets: the public OPPORTUNITY dataset, the Sony Smartwatch Gesture dataset and a custom Samsung Galaxy watch 6 dataset. The proposed framework leverages five step sequence-to-label problems which are stepwise posed as data streams transforming raw IMU data into trainable tensors. For that purpose, a deterministic seven-step transformation consists of transport-agnostic ingestion, gap interpolation, low-pass filtering, and on-the-fly label-uncorrupted znormalisation. Z-normalised epochs of 100 frames at sampling frequency 50 Hz are fed to five sequence-deep architectures. The BiLSTM, GRU, 1D CNN, CNN+LSTM, and a four-block Temporal Convolutional Network TCN were trained and evaluated under stratified 80/20, Leave-One-Subject-Out (LOSO), Leave-One-Experiment-Out (LOEO), and stratified K-fold validation—all separately on each dataset. The TCN consistently achieves the best trade-off between accuracy and efficiency: a macro-F1 score of 0.923 on the OPPORTUNITY 80/20 split, 0.967 under LOSO on the Sony corpus, and 0.427 under LOSO on OPPORTUNITY using only 200k parameters while sustaining sub-20 ms inference on standard CPUs. A Bayesian hyperparameter search on the SJSU HPC cluster demonstrates a 40% reduction in tuning time, showcasing scalable reproducibility across 1,232 CPU cores and multiple GPU nodes.

Download

Available for download on Wednesday, May 20, 2026

Share

COinS