The most inaccurate indicators measured by smart watches are actually these...

From heart rate, blood pressure, sleep to energy consumption, fatigue status, and maximum oxygen uptake, smart watches and bracelets display more and more indicators, but it feels like some values ​​are not accurately measured?

Your feeling is correct. The accuracy of various indicators varies greatly. Some are so accurate that doctors will use them for reference, while others are only suitable for casual viewing.

The following table summarizes the accuracy of some common indicators, with the highest accuracy being 5 stars and the lowest being 1 star.

This table takes all the conditions into consideration for rating. The algorithms of different manufacturers may vary, and the rating is somewhat subjective. 5 stars means very good accuracy (measured by the gold standard), 4 stars means good, 3 stars means medium, 2 stars means poor, and 1 star means very poor.

None of the indicators here reach 5 stars, which means that no data is absolutely accurate. Why is it inaccurate? It may be related to the measurement method, sensor, algorithm, wearing and interpretation method.

Next, we fill in more detailed information in the table: why it is accurate or inaccurate, how accurate it is, and how to measure it more accurately. At the end of the article, you will get a table with twice the amount of information, as well as the ability to judge how to use each indicator.

Whether it is accurate or not depends on the indicator.

Estimated or newly created

Nowadays, a smart watch weighing dozens of grams can integrate nearly 10 sensors, such as photoelectric sensors for measuring heart rate, GPS sensors for measuring longitude and latitude, as well as air pressure, temperature, acceleration sensors, etc.

Wearable devices (smart watches, bracelets, rings, etc.) measure directly through sensors, showing us a limited number of basic indicators (the original data also needs algorithm processing, so it is written as direct measurement here for ease of understanding); these indicators are integrated and calculated, and then endless new indicators are continuously produced. In other words, as long as physiology and sports physiology are used as the basis, a few basic indicators can be reversed to get a bunch of indicators.

There are more and more indicators, but are they all reliable? As long as there is measurement, errors are inevitable, but most indicators have a measurement method that is recognized to have the smallest error, which is generally called the "gold standard". For example, the gold standard for measuring heart rate is electrocardiogram, the gold standard for measuring sleep time and stage is polysomnography, and the gold standard for measuring energy consumption is the double-labeled water method.

Gold standards are usually measured under laboratory conditions. Most of the equipment is expensive, the measurement steps are complicated, and the assistance of experienced operators is required. Currently, none of the indicators provided by smart watches, bracelets or rings are measured by gold standards. Therefore, none of the indicators in the table at the beginning of the article can get 5 stars. At the expense of some data accuracy, wearable devices have brought more convenient and lower-cost measurement methods.

The gold standard for measuring heart rate is electrocardiogram; smart watches can measure heart rate continuously, which is simple but less accurate丨medpick/Sina Public Test

Among the gold standard indicators, some data are directly measured or obtained through simple calculations, such as measuring heart rate through photoelectric sensors and calculating pace through exercise distance and time.

Some data is estimated by algorithms based on directly measured data, such as estimating energy consumption from heart rate and accelerometer data. Different manufacturers may have different algorithms, and the same manufacturer's algorithm may be constantly improved, so the results may vary greatly. In most cases, the estimated data is not as accurate as the direct measurement.

For other indicators that do not have a gold standard, it can be said that most of them are not very accurate. Many of these indicators only exist in the concepts of sports science (such as load, fatigue, and recovery), and cannot be accurately measured. Sometimes they are based on subjective feelings. Some indicators even have no scientific definition and are created by the "arms race" between manufacturers.

How big is the gap?

Just compare it with the gold standard.

If you want to know how accurate the indicator is, you can measure it with a wearable device and the gold standard, and then compare the results.

In fact, most manufacturers do the same, but they generally won’t say how big the gap is. However, by analyzing how the data is obtained and reading the articles published by researchers, you can still get a general idea of ​​the accuracy of the data.

1

Heart rate is one of the most accurate indicators

Heart rate is related to many health and sports related indicators. Watches and bracelets can directly display heart rate and can also provide many indicators based on heart rate estimation. Therefore, the accuracy of heart rate measurement determines the accuracy of many other indicators.

The gold standard for measuring heart rate is the electrocardiogram, which detects the heart's electrical activity and measures heart rate using electrodes placed on the chest and limbs.

When wearable devices display heart rate continuously, the measurement method is usually photoplethysmography (PPG), which can be affected by many factors, such as exercise intensity, exercise type, wrist movement, wristband tightness, skin pigmentation and surface dirt, and arrhythmia.

According to comprehensive tests of 18 studies, heart rate measurement is more accurate at rest or during low-intensity exercise. As the intensity of exercise increases, the probability of measuring data and the reliability of the data will decrease significantly. In an analysis of 249 studies, the average error of heart rate measurement was ±3%.

Therefore, when the wearable device displays stable values ​​in a quiet state, the heart rate data is relatively reliable and can be used to help judge health and exercise conditions. The data accuracy decreases during strenuous exercise. If you want to get more accurate data, you can wear a chest heart rate belt.

Chest heart rate belt. Copyright image. Reprinting and using may cause copyright disputes.

2

Sleep, total time slightly better than stages and quality

Some people check last night’s sleep indicators as soon as they wake up. They may feel they slept well, but when they see a lower overall score, they suddenly feel tired. In fact, there is no need to worry.

The gold standard for measuring sleep is polysomnography, which refers to the simultaneous measurement of multiple signals, including electroencephalogram, electrocardiogram, oculogram and electromyogram. After obtaining the raw data, sleep experts will combine the results to derive the sleep time and manually score to analyze the sleep stages.

Polysomnography diagram | verywell

Wearable devices evaluate sleep by measuring heart rate and wrist activity (accelerometer), calculating indicators such as heart rate variability and respiratory rate, and then combining them with background information such as age, height, weight and gender, based on a neural network model, to ultimately obtain bedtime and waking time, sleep start and end time, total sleep duration and sleep latency, wakefulness duration, duration and proportion of each sleep stage, and an overall sleep score based on this information.

Wearable devices evaluate sleep patterns | Photo provided by the author

From the measurement method point of view, if you remain motionless for a long time before falling asleep, it may be mistakenly judged as falling asleep , which may overestimate the total sleep time.

The specific algorithms of different brands are inconsistent, resulting in different errors. A review article on the application of wearable technology in sleep mentioned that compared with polysomnography, watches perform relatively well in estimating total sleep time, with an overall accuracy of about 70% to 90%; they perform poorly in measuring sleep stages , with an accuracy of about 50% to 90% for light sleep judgment and about 30% to 80% for deep sleep and rapid eye movement sleep.

As for the overall sleep score, there is no corresponding score in medicine. When doctors evaluate sleep quality and provide treatment, they will analyze many indicators such as the time to fall asleep, sleep time, efficiency, abnormal state, hypnotic drugs, daytime life and work status, etc.

For the estimated indicators such as sleep, the more accurate ones can be used as a reference, such as the total sleep time, and the other indicators should not cause anxiety. If the whole person is in good condition, there is no need to worry about the low total sleep score . If you always feel that you can't sleep well, you can go to the hospital for polysomnography to detect problems in time.

3

Recovery, one of the least accurate indicators

The above indicators all have gold standards, while some indicators do not have gold standards and are created based on a certain theory, such as recovery status.

To make progress in training, you need to continuously increase the training stress without reaching the line of overtraining, so it is very important to measure and detect recovery. However, recovery is a very comprehensive and complex indicator , which is affected by training (training volume, type, intensity, etc.), non-training (work, interpersonal relationships, illness, drugs, etc.) and recovery (sleep, diet, recovery time, recovery methods, etc.) factors.

Recovery is affected by many factors, including training, sleep, diet, etc.

Autonomic nervous system activity is an important indicator of how the body is experiencing stress and recovery. When the body is stressed, it is usually physiologically manifested as increased sympathetic nervous system activity and decreased parasympathetic nervous system activity, while recovery is the opposite. Studies have shown that heart rate variability is a powerful tool for analyzing the interaction between the sympathetic and parasympathetic nervous systems.

Since there is no gold standard, some wearable device manufacturers use weighted models to estimate recovery. The specific method is to collect a series of indicators that may affect recovery, such as heart rate, sleep and training, calculate heart rate variability, respiratory rate, oxygen consumption and other data, and then weight the sum of different indicators according to sports science principles. The value obtained represents the recovery status.

The disadvantage of this approach is that it cannot account for all influencing factors . For example, the physiological cycle and interpersonal relationships can affect recovery, but may not be calculated by the model, resulting in differences between the displayed data and the actual status, and insufficient or excessive training when using the data as a guide.

However, it makes sense for manufacturers to introduce such indicators, because not everyone has enough knowledge to analyze and interpret the above raw data related to recovery. Sacrificing some accuracy and making some simple assumptions (such as less sleep and more activity equals poor recovery), a recovery score reminder may be much better than letting people look at complex physiological data.

How should these indicators be used?

According to the classification method at the beginning of the article, all indicators can be divided into three categories: measurement, estimation, and creation.

The measured indicators usually have a small error, such as heart rate, distance, heart rate variability and pace. These indicators are relatively reliable and can be used as a reference for observing health conditions, adjusting lifestyles and exercise plans. For example, if your heart rate is higher than usual when you wake up this morning, is it because you didn't sleep well last night? Or have you over-exercised recently? Should you reduce the amount of exercise or take a day off?

Estimated indicators are derived through algorithms based on measured indicators, such as sleep, energy expenditure, and oxygen uptake. At this time, the measurement error superimposed on the algorithm error may reduce the accuracy of the estimated indicators. When interpreting such indicators, you need to be more cautious. For example, the overall sleep score is sometimes consistent with the level of sleepiness, and sometimes there is a gap; the energy expenditure of walking estimated by wearable devices may be more accurate, but the consumption of resistance (strength) training may be underestimated.

The above two are gold standard indicators. Even if the current measurements are not very accurate, we can expect that the measurement technology or algorithm will improve so that the data will get closer and closer to the accurate value.

The created indicators are some indicators that do not have a gold standard for measurement. They are created by the algorithm based on the first two types of indicators according to some definitions or ideas, such as recovery status, training effect, etc. Since there is no gold standard for comparison, and the sensor hardware and algorithms of different manufacturers are inconsistent, and the algorithm of the indicator is not public, it is difficult to verify the accuracy of the data.

Therefore, for these created indicators, we do not need to worry too much about the absolute values ​​of the numbers. We can more proactively understand the body's response to daily life and exercise by understanding the changing trends of the indicators and combining them with our own subjective feelings.

In addition, device manufacturers will regularly release software updates. Checking and installing these updates in a timely manner can ensure that the device always uses the latest algorithms, which can improve the accuracy of the indicators to a certain extent.

The last table summarizes the important points of the whole article. Referring to it to interpret the data provided by wearable devices may help you reduce some confusion and increase control over your health and exercise.

References

[1]Altini M, Plews D. What is behind changes in resting heart rate and heart rate variability? A large-scale analysis of longitudinal measurements acquired in free-living[J]. Sensors, 2021, 21(23): 7932.

[2]Cudejko T, Button K, Al-Amri M. Validity and reliability of accelerations and orientations measured using wearable sensors during functional activities[J]. Scientific reports, 2022, 12(1): 14619.

[3]Shei RJ, Holder IG, Oumsang AS, et al. Wearable activity trackers–advanced technology or advanced marketing?[J]. European Journal of Applied Physiology, 2022, 122(9): 1975-1990.

[4]Miller DJ, Sargent C, Roach G D. A validation of six wearable devices for estimating sleep, heart rate and heart rate variability in healthy adults[J]. Sensors, 2022, 22(16): 6317.

[5]Germini F, Noronha N, Borg Debono V, et al. Accuracy and acceptability of wrist-wearable activity-tracking devices: systematic review of the literature[J]. Journal of medical Internet research, 2022, 24(1): e30791.

[6]Li YI, Zhong-Hua LV, Shun-Ying HU, et al. Validating the accuracy of a multifunctional smartwatch sphygmomanometer to monitor blood pressure[J]. Journal of Geriatric Cardiology: JGC, 2022, 19(11): 843.

[7]de Zambotti M, Goldstein C, Cook J, et al. State of the science and recommendations for using wearable technology in sleep and circadian research[J]. Sleep, 2023: zsad325.

[8]https://www.firstbeat.com/en/athletes-recovery-analysis-firstbeat-white-paper-2/

[9]https://www.firstbeat.com/en/firstbeat-white-paper-sleep-analysis-method-based-on-heart-rate-variability/

[10]Doherty C, Baldwin M, Keogh A, Caulfield B, Argent R. Keeping Pace with Wearables: A Living Umbrella Review of Systematic Reviews Evaluating the Accuracy of Consumer Wearable Technologies in Health Measurement. Sports Med. 2024 Jul 30. doi: 10.1007/s40279-024-02077-2. Epub ahead of print. PMID: 39080098.

Planning and production

Source: Guokr (id: Guokr42)

Author: ZIYI

Editor: Yang Yaping

Proofread by Xu Lai and Lin Lin

The cover image and the images in this article are from the copyright library

Reprinting may lead to copyright disputes