Assessing Musculoskeletal Health with Wearables and AI

Reading time:

time

min

October 19, 2023

Appsilon teamed up with Prof. Agnieszka Szczęsna from Silesian University of Technology and Dr. Monika Błaszczyszyn from Opole University of Technology to determine whether smartphones could be used to detect symptoms of frailty in elderly adults. Using data gathered from inertial measurement unit (IMU) sensors, we developed a proof of concept suggesting that artificial intelligence could be used for such evaluations and hence used for early disease detection and prevention in the elderly population.
<h2>Monitoring Health with Smart Watches and Phones</h2>
Most of the world’s population uses smartphones, and a growing number of people utilize wearable devices such as smartwatches or smartbands, coupled with apps for fitness tracking. These devices are equipped with various sensors that measure heart rate, blood oxygen levels, sleep phases, and counting steps, among other variables. Many of these measurements, like the step counter, are not measured directly. Instead, they rely on data from an IMU“inertial measurement unit” (IMU), typically consisting of so-called “6DOF” sensors, including an accelerometer that measures linear acceleration in three dimensions and a gyroscope that measures angular velocity.

There is no straightforward way to measure steps from this raw data, especially since these sensors are typically worn on the wrist, not on the foot. Therefore, these devices employ various machine learning algorithms to estimate when individual steps occur.

While these sensors might have a great impact on your health by giving you the motivation to take that walk you’ve been postponing, data from wearable sensors has an even greater potential to contribute to health diagnoses and disease prevention.

In this article, we’ll discuss one such application and the power of modern solutions for geriatric medicine.
<h2>What is Frailty?</h2>
An area where early diagnosis and regular monitoring is important is assessing the health of the elderly. Very often, functional decline and mobility impairments indicate either impending diseases, frailty, or increased risk of falling. Early diagnosis allows effective prevention and increases the chances of effective rehabilitation, ultimately improving the quality of life and the fitness of elderly patients.

Frailty is defined as a syndrome of simultaneously occurring factors, consisting of motor slowness, fatigue, weakness, cachexia, and low activity. People with frailty syndrome are prone to the occurrence of diseases, various types of dysfunctions, including falls, reduced quality of life, and even premature mortality.
<h2>Measuring Frailty of Elderly People</h2>
The “Timed Up & Go” (TUG) test has been demonstrated in prior studies to be an accessible, valid, and reliable instrument for evaluating physical capability, distinguishing between older adults who are frail and those who are not by capturing a variety of age-related physiological changes. The TUG test is one of the tests which can be used for estimating frailty. The participant sits in a chair and is asked to stand up, approach an obstacle 3 meters ahead of them, go around it, come back, and sit down. The time it takes a person to execute one TUG is measured, and if it takes more than 6 seconds, it is considered a symptom of frailty (a cutoff that varies from 5.0s to 7.7s, depending on the source).

<img class="size-full wp-image-21306" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b01a0cfeabe8f16c2f2c5e_Peach-3-Panel-Collage-Beauty-Skin-Steps-Instagram-Post-_1_.webp" alt="IMU sensors were mounted on wrists, ankles and hips of the participants." width="1080" height="1080" /> IMU sensors were mounted on wrists, ankles and hips of the participants.

<video width="100%" height="auto" src="https://wordpress.appsilon.com/wp-content/uploads/2023/10/video-1.webm" loop="true" autoplay="true" controls="true"></video>

Unfortunately, many such tests need to be carried out to estimate frailty accurately. Thus, despite the fact that they are also easy to implement, regular assessments are costly, as they require significant time from physicians and patients to conduct the tests and the diagnosis. Furthermore, good performance in test conditions does not always translate to good performance in everyday life, and all of these performance evaluations are criticized for their lack of specificity.

Thus, it would be beneficial to develop assessments that could be carried out by the patients themselves, on a regular basis, in everyday life. These should take into account not only gait speed, but other kinematic patterns indicating or preceding development of frailty. How can we achieve that?
<h2>Can AI be Used to Measure Frailty?</h2>
One idea would be to use data from IMU sensors, such as ones embedded in your smartwatch, or even your smartphone, to estimate frailty. An experienced doctor can often infer frailty of participants just from looking at the way they walk, for example, during the TUG test. This is not a precise measurement, but could AI help with that?

We teamed up with Prof. Agnieszka Szczęsna from Silesian University of Technology and Dr. Monika Błaszczyszyn from Opole University of Technology to try to figure this out. The goal is to verify whether data from various IMU sensors (placed on wrists, hips, ankles) could be analyzed by machine learning models to infer the frailty index of the participants. Dr Błaszczyszyn conducted the experiments, in which she recorded data from IMU sensors for a participant who executed a TUG test, and independently they assessed their frailty on a scale of 0-2. In total they obtained 653 records of the sensor data from the TUG tests. Our goal is to infer the frailty index from these sensor data and some basic information about participants (age, weight etc.).
If feasible, this could pave the way for more accurate and more timely diagnoses, either using sensors attached to bands, or even consumer hardware containing IMUs.
<h2>Teaching AI How Humans Move</h2>
When we got the data, we took a close look at it. In the plots of sensor measurements, we notice that the sensor data does have meaningful features:

<img class="size-full wp-image-21310" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b01a0d2d3baed3c5a571dc_photo_2023-10-19_14-01-31.webp" alt="Figure 1. Measurements from one of the gyroscopes located on hips clearly shows all of the stages of TUG: standing up, walking, going around the obstacle, rotating before sitting down, and sitting down. Colors distinguish measurements for various participants, and time has been normalized to [0,1]" width="1257" height="504" /> Figure 1. Measurements from one of the gyroscopes located on hips clearly shows all of the stages of TUG: standing up, walking, going around the obstacle, rotating before sitting down, and sitting down. Colors distinguish measurements for various participants, and time has been normalized to [0,1] Indeed, in the graph of gyroscope measurements, we easily distinguish various stages of the experiment happening, as marked in the graph above (Fig. 1). We can even distinguish whether the person passes the obstacle on the left or on the right, and in which direction they rotate before sitting down! That convinced us there is meaningful information in these measurements, and we proceeded to design some models that could detect frailty from it.

Moreover, as Prof. Szczęsna and Dr. Błaszczyszyn note, “<i>Duration of rotation is one important measure that differentiates fall risk and frailty syndrome.”</i> They stress that various strategies that people choose during the rotations might very well be indicators of frailty, for example, how close one puts their feet during the rotation, or how much one slows down before performing it. Potentially, one could detect such behaviors from the sensor data.
<h3>Feature Engineering Using FFT+PCA</h3>
Firstly, we tried some “classical” approaches, partially hand-engineering features which could be used to infer frailty, trying to quickly quantitatively verify whether sensor data contains information meaningful for estimating frailty. We computed the mean absolute deviation (MAD) and standard deviation (STD) statistics for the sensors located on the hips, and built a random tree classifier on top of that and other features that are easy to acquire (age, height, mass, gender). We decided to, for this proof-of-concept, only predict whether someone has developed symptoms of frailty (score of 1-2) or not (score of 0), as the classes were very imbalanced - there were 10x more people with scores 0-1 than with score 2. On this task, the model achieved an <b>accuracy of 61%</b> on the test set. We also experimented with computing the fast Fourier transform (FFT) of the signals and extracting the most important frequency vectors using principal component analysis (PCA). Together with basic patient data, we achieved an accuracy of 62%, slightly but not significantly better. These models definitely do not achieve the performance needed for deploying such a model, but they do demonstrate that sensor data does contain information meaningful for the assessment of frailty.
<h3>Employing Neural Networks</h3>
Since we obtained quite a lot of sensor data (over 5 hours of recordings of sensor measurements in total), we decided to also test whether neural networks can achieve even better performance. Using data from the hips sensor and basic participant data (age, height, mass), we built a simple convolutional neural network which achieved an accuracy of 63%. This is a somewhat better result than the one obtained utilizing features from FFT+PCA as described in the previous section, which further proves that sensor data contains features important for predicting frailty. By manipulating the model structure and adjusting hyperparameters, we hope to further improve its performance and validate whether sensor data (together with basic participant info like age, weight, etc.) is sufficient for assessing frailty.
<h2>New AI Approaches to Health Monitoring</h2>
Our results so far suggest that data from IMU sensors contain features that could be used to assess frailty. If we can achieve high accuracy using these methods, it will suggest that using a single IMU unit located in a fitness band, or even a smartwatch held in a pocket, could be used for monitoring the health of elderly people. <b>That could be a breakthrough for early detection of mobility impairments</b>.

While gait speed in the elderly population is one of the best documented factors of frailty syndrome, it is essential for clinical practice to better understand gait changes. Such understanding would provide a basis for clinicians to accurately assess gait disorders based on the study of kinematic patterns. The identification of gait abnormalities preceding frailty would provide a basis for designing exercise programs or devices helping to improve gait, helping to prevent the development of frailty syndrome. It seems that our models could bring invaluable insights to the researchers in that area.

Potentially, with the use of wearable sensors, many diseases can be diagnosed early on or prevented entirely. Researchers in medicine lead multiple efforts to advance methods for early diagnoses, with data from wearables playing a key role. Our AI team at Appsilon is skilled at quickly checking whether a problem can be efficiently solved using machine learning, and advancing technology to improve the performance of existing models. Experiments like this often lead to solutions that the researchers look for, as it was with <a href="https://appsilon.com/monitoring-marine-ecosystems-with-machine-learning/" target="_blank" rel="noopener">plankton health monitoring</a>, <a href="https://appsilon.com/yolo-counting-nests-antarctic-birds/" target="_blank" rel="noopener">counting cormorant nests</a>, or <a href="https://appsilon.com/data-for-good/mbaza-ai/" target="_blank" rel="noopener">monitoring biodiversity in natural habitats</a>.
<h2>Join Us in Shaping the Future of Healthcare</h2>
Our Machine Learning Team at Appsilon plays a pivotal role in providing solutions to solve complex healthcare problems. In this post, we have applied ML to transform raw data from wearables into meaningful insights, which could aid medical professionals in their diagnostic journey.

Are you intrigued by the possibilities offered by wearable sensors and AI in healthcare? Eager to explore innovative solutions that can redefine the future of medicine? Take the first step forward shaping the future of healthcare. <b>Connect with us today and let’s explore the boundless possibilities together</b>.