Optimization of Swim Pose Estimation and Recognition with Data Augmentation

Publication Date

1-1-2024

Document Type

Conference Proceeding

Publication Title

Proceedings of the IEEE Southwest Symposium on Image Analysis and Interpretation

DOI

10.1109/SSIAI59505.2024.10508644

First Page

101

Last Page

104

Abstract

Swim pose estimation and recognition is a challenging problem in machine learning and artificial intelligence as the body of the swimmer is continuously submerged under the water. The objective of this paper is to enhance existing ML models for estimating swim poses and for recognizing strokes to aid swimmers in pursuing a more perfect technique. We developed a novel methodology augmenting raw video data and adjusting a YOLOv7 base model to enhance swim pose estimation. We found the standard multi-class classification using a CNN to be insufficient for stroke recognition due to the similarity between strokes, so we designed a hierarchical binary classification tree using multiple ensembles of multilayer perceptron (MLP), CNN, and residual network (ResNet) models. Through these optimizations, the confidence level of pose estimation has increased by over 30%, and the ensembles of our recognition model have achieved approximately 80% accuracy. Fine-tuning of our recognition models and research combining joint keypoint coordinates with angle measurements as inputs could further increase the accuracy of our models.

Department

Electrical Engineering

Share

COinS