Composite Human Activity Recognition Utilizing Knowledge Distillation and Sensor Fusion Focusing on Resource Constrained Microcontrollers

Publication Date

3-1-2026

Document Type

Article

Publication Title

Expert Systems with Applications

Volume

298

DOI

10.1016/j.eswa.2025.129652

Abstract

This study presents a cost-effective, low-computation system for composite Human Activity Recognition (HAR) that leverages knowledge-distilled neural networks on a Microcontroller Unit (MCU) to minimize reliance on cloud processing. A key contribution of this work is the investigation of plantar pressure sensor data within a knowledge distillation framework, addressing a notable gap in the existing literature. The proposed solution centers around the ESP32-S3 DevKit C1, equipped with a dual-core 240 MHz Tensilica chip, 320 KiB of usable Static Random Access Memory (SRAM), and built-in Wi-Fi and Bluetooth. Significantly, both the teacher and the student models surpass existing state-of-the-art methods, achieving F1-scores of 99.33 %, 98.36 %, and 97.68 % respectively, in classifying a comprehensive set of 21 activities (15 composite and 6 simple). The distilled student models demonstrate remarkable efficiency, with execution times of 1.83 and 0.64 s, memory footprints of only 62 KB and 82 KB, and flash memory usage of approximately 209 KB and 127 KB, while maintaining low power consumption of 210 mW and 215 mW, respectively. Furthermore, we have developed an end-to-end prototype that integrates the ESP32-S3 with a WitMotion Inertial Measurement Unit (IMU) sensor. This system autonomously manages data acquisition, feature extraction, and inference in under 7 s with a total power consumption of approximately 295 mW.

Keywords

Composite human activity recognition, Knowledge distillation, Low-cost, Microcontrollers, Nural networks, Resource constrained AI, TinyML

Department

Computer Engineering

Link to Full Text

Share

COinS