Empowering Hand Rehabilitation with AI-Powered Gesture Recognition: A Study of an sEMG-Based System

Overview

Introduction

Stroke has become a major contributor to human death globally, with over 10 million people suffering from it annually. This debilitating disease affects the lives of patients and their families, significantly reducing the quality of life of all involved. Despite progress in stroke treatment and increased survival rates in recent years, there is still a critical need for advanced rehabilitation methods to speed up recovery and improve motor function in post-stroke patients. This is where the concept of assistive devices in rehabilitation comes into play. Stroke can cause severe damage to the central nervous system, resulting in various physical impairments and limitations. One of the most common injuries in stroke patients is the loss of wrist motion, a crucial joint connecting the hand and the arm. As such, rehabilitation training for patients with stroke and wrist injuries is essential for improving their overall health and well-being. With the help of specialized rehabilitation techniques and assistive devices, stroke patients can regain some of their motor abilities and improve their quality of life.

This article presents a hand rehabilitation system with a soft glove that supports mirror, task-oriented therapies, and neural plasticity. The actuated glove, integrated with a linear actuator, provides the driving force for the affected hand during rehabilitation training. The sEMG sensor enables cooperation between both hands and was used to control the actuated glove to let the affected and the non-affected hands move simultaneously. An integrated sensor measures the bending angle and the progress of the rehabilitation programmer. A machine learning algorithm was developed to classify the sEMG gestures as a control command for the actuated glove. This gives the affected hand the suitable driving force to perform the corresponding motions.

Materials and Methods

Hand Design and Actuation 

The research and development of hand rehabilitation equipment are still in their infancy. Most exoskeleton-hand robots focus on improving finger joints and increasing muscle strength, ignoring the vital role of active brain participation and the sensory function input into the motor function. Therefore, the effect of fine motor rehabilitation on hand function is not considered. At present, the hand-function rehabilitation robots design adopts an exoskeleton structure, which mainly provides a driving force through motors and pneumatics to complete flexion and extension of the fingers and joints. Four main driving methods have been used for hand-function rehabilitation robots. They mainly include a motor drive, a pneumatic artificial muscle drive, a memory alloy drive, and a lasso drive. Because the lasso drive has good flexibility and linearity, a lasso drive was used to design hand-functional rehabilitation robots.

Recognition Methods of the sEMG Signals 

The electrical voltages in sEMG signals range from −5 to +5 (mV) and are influenced by both the movements performed and the muscle extraction and contraction level. The continuous availability and variability of the signal can be measured using a suitable detection element. These signals have significant potential in rehabilitation, where they can be combined with an appropriate recognition system. However, given that the sEMG signals traverse multiple tissues and muscles before being acquired from human muscles, they are susceptible to interference from crosstalk, obstructions, and noise. Different machine learning and A.I. techniques have been used to classify and recognize the EMG signals. The classifier can be a Supported Vector Machine (SVM), the k-nearest neighbor (kNN) algorithm, linear discriminant analysis (LDA), and a neural network (N.N.), with different classification accuracy (C.A.) and complexity for different methods.

Design and Manufacture of the Hand Robot

This study aimed to introduce a wearable rehabilitation glove based on flexible lasso transmission driven by a flexible rope and a linear actuator. The glove was designed to assist patients with hand impairments in the rehabilitation process and enable them to perform routine activities in daily life, which can boost their confidence and promote their independence. As shown in Figure 1, the glove can simulate the functions of the human hand and facilitate the recovery of patients with various hand injuries or conditions.

Figure 1. Flexible Hand Rehabilitation Robot with a Linear Actuator and an sEMG Sensor. Adapted from source

Glove and Finger Structure 

The structure of the wearable glove used for hand rehabilitation is shown in Figure 4. The outer layer of the glove is composed of a spring tube that remains stationary, while the inner layer is a cored wire with a steel wire lining, which drives the movement of the fingers through the movement of the cored wire. The design incorporates two groups of ropes for each finger, one to drive the finger to bend on the inner side, and the other to drive the finger to straighten on the dorsal side. By pulling the ropes, the glove mimics the natural movement of the fingers, allowing the wearer to perform a range of exercises to improve the motor function. A pneumatic tube was used to reduce friction between the cored wire and the outer spring sleeve. A wire sleeve-fixing structure was used inside the palm. Considering the interference with finger movement, the casing-fixing device and flexible transmission connection was designed as shown in Figure 2.

Figure 2. Flexible transmission and glove with sensors. Adapted from source

Results

Accuracy of Gesture Recognition

This article used the PyQt5 framework to build software for EMG acquisition and raw data storage. In real time, the software effectively displayed the four lead EMG signals alongside the real-time predicted gestures, providing accurate results to the users. The 1D-CNN algorithm and the Inception Time algorithm was adopted. Figure 3 shows the accuracy achieved by using the two algorithms and the accuracy for specific gestures. In Figure 14a, the 1D-CNN algorithm was adopted, achieving an accuracy of 89.52% in the training set and of 76.84% in the verification set, with an overall accuracy of 80.98%. In Figure 14b, the Inception Time algorithm was adopted, achieving an accuracy of 91.60% in the training set and of 90.09% in the verification set, with an overall accuracy of 90.89%.

Figure 3. Confusion matrix and iteration accuracy of different algorithms, (a) using the 1D-CNN algorithm, (b) using the Inception Time algorithm. Adapted from source

Control of the Hand Rehabilitation Robot Based on sEMG

The sEMG drive of the hand rehabilitation robot using the framework is shown in Figure 4. Based on the above framework, an online recognition software was developed for grasping and extending a hand using the above algorithm. The control of the rehabilitation robot based on this interface was implemented.

The algorithm was developed with the help of the eight people, and data collection was completed after setting gesture sequences and collection methods. Subsequently, data filtering and other pre-processing operations were carried out, and relevant features in the time and frequency domains were extracted, ultimately completing the development of the algorithm. For the online control of robots, the first step is to select the mirror mode through a screen, and the computer and device communicate through the TCP protocol. After obtaining electromyographic data through the computer software, the data were processed and algorithmically recognized, ultimately outputting the motion instructions for the robot. The complete hand rehabilitation robot was constructed as described in this study is shown in Figure 5.

Figure 4. Framework of communication and control. Adapted from source

Figure 5. Hand rehabilitation robot that was developed in this study. Adapted from source

Conclusion

In conclusion, the presented wearable hand rehabilitation system has the potential to significantly impact the motor recovery of paresis and hemiparesis patients. Actuated gloves with sEMG sensing are a flexible, wearable technology that is safe, comfortable, and portable. Additionally, the system can be affordable for low- and middle-income countries, which have limited access to rigid exoskeleton devices due to their high cost. Compared to dedicated-data gloves with targeted sensors, the wearable glove utilizing biomedical signals offers improved signal quality and higher accuracy in detecting the desired motions during rehabilitation training. This connection between the desired motion activated by the brain and sent to the muscle and the actual movement executed by the actuated glove shown on a screen can speed up the rehabilitation process based on the mirror therapy technique. However, it is worth noting that achieving a fine-grained classification of the training gestures still requires the precise placement of the EMG electrode. This study presents a promising wearable rehabilitation system for hand motor recovery that can be used in various settings, including clinical and home-based environments

Adapted from:

  1. Guo K, Orban M, Lu J, Al-Quraishi MS, Yang H, Elsamanty M. Empowering Hand Rehabilitation with AI-Powered Gesture Recognition: A Study of an sEMG-Based System. Bioengineering (Basel). 2023 May 6;10(5):557. doi: 10.3390/bioengineering10050557. PMID: 37237627; PMCID: PMC10215961