Deep Reinforcement Learning for EMG-based Control of Assistance Level in Upper-limb Exoskeletons
Publication Date
1-1-2022
Document Type
Conference Proceeding
Publication Title
2022 International Symposium on Medical Robotics, ISMR 2022
DOI
10.1109/ISMR48347.2022.9807562
Abstract
In this paper, we propose a deep reinforcement learning (DRL) method to control the assistance level of an upper-limb exoskeleton in real-Time based on the electromyo-graphic (EMG) activity of human muscles in 3D point-To-point reaching movements. The proposed autonomous assistive device would enhance the force exertion capability of individuals by resolving major challenges such as identifying scaling factors for personalized amplification of their effort and not requiring lengthy offline training/adjustment periods to perform their manual tasks comfortably. To this end, we employed the Twin Delayed Deep Deterministic Policy Gradient (TD3) method for rapid learning of the appropriate controller's gain values and delivering personalized assistive torques by the exoskeleton to different joints to assist the wearer in a weight handling task. A nonlinear reward function is defined in terms of the EMG activity level and the position deviation from the destination point to simultaneously minimize the muscle effort and maximize the positioning accuracy. This facilitates autonomous and individualized physical assistance by rapid exploration of reward values and adopting various action gains within a safe range to exploit the ones that maximize the reward. Based on experimental studies on an exoskeleton with soft actuators that we have developed, the proposed DRL method is able to identify the most appropriate assistive gain for each joint of the exoskeleton in real-Time for the user with a fast rate of convergence (during the first two minutes). Optimum assistive gains are identified for each degree of freedom (DOF) in a 4 kg weight handling task in 3D space, which required less than 15% of the muscle contraction level (EMG activity).
Funding Sponsor
Canadian Institutes of Health Research
Keywords
actor-critic method, assistive exoskeleton, Deep reinforcement learning (DRL), EMG-based control, twin delayed deep deterministic policy gradient (TD3)
Department
Mechanical Engineering
Recommended Citation
Martin Oghogho, Mojtaba Sharifi, Mia Vukadin, Connor Chin, Vivian K. Mushahwar, and Mahdi Tavakoli. "Deep Reinforcement Learning for EMG-based Control of Assistance Level in Upper-limb Exoskeletons" 2022 International Symposium on Medical Robotics, ISMR 2022 (2022). https://doi.org/10.1109/ISMR48347.2022.9807562
