How Accurate Are Wearable Fitness Trackers? Less Than You Might Think

In 2010, Gary Wolf, then an editor at Wired magazine, gave a TED talk in Cannes titled “The Quantified Self.” It addressed what he called “the new fad” among tech enthusiasts: Early adopters using gadgets to monitor everything from physiological data to mood to even the number of diapers their children were using.

Wolf acknowledged that these individuals are exceptions — tech geeks fascinated by data — but their behavior has permeated mainstream culture.

From smartwatches that track our steps and heart rate to fitness trackers that log sleep patterns and calories burned, these gadgets are now ubiquitous. Their popularity is emblematic of the modern obsession with quantification—the idea that if it’s not being recorded, it doesn’t count.

At least half the people in a room are likely wearing a device, such as a fitness tracker, that measures some aspect of their life. Wearables are being adopted at a rate reminiscent of the cellphone boom in the late 2000s.

But measuring your own movement still raises an important question: Do wearable devices really measure what they promise?

Together with my colleagues Maximus Baldwin, Alison Keogh, Brian Caulfield, and Rob Argent, I recently published an umbrella review (a systematic review of systematic reviews) examining the scientific literature on whether consumer wearable devices can accurately measure parameters such as heart rate, aerobic capacity, energy expenditure, sleep, and step count.

On a superficial level, our results were quite positive. Accepting some error, wearables can measure heart rate with a margin of error of plus or minus 3%, depending on factors such as skin tone, exercise intensity, and type of activity. They can also accurately measure heart rate variability and have good sensitivity and specificity in detecting arrhythmia, a problem with a person’s heart rate.

In addition, they can accurately estimate what is known as cardiorespiratory fitness, which is how well your circulatory and respiratory systems deliver oxygen to your muscles during physical activity. This can be quantified by something called VO2Max, which is a measure of how much oxygen your body uses during exercise.

The ability of wearables to accurately measure this is improved when these predictions are generated during exercise (rather than at rest). In the realm of physical activity, wearables typically underestimate step counts by about 9%.

A difficult undertaking

However, the discrepancy was larger for energy expenditure (the number of calories burned during exercise), with margins of error ranging from -21.27% to 14.76%, depending on the device used and the activity undertaken.

The results weren’t much better for sleep. Wearables tend to overestimate total sleep time and sleep efficiency, typically by more than 10%. They also tend to underestimate sleep latency (the delay in falling asleep) and wakefulness after sleep onset. Errors ranged from 12% to 180%, compared to the gold standard measurement used in sleep studies, known as polysomnography.

As a result, despite the promise of wearable devices, we have found that conducting and synthesizing research in this area is very difficult. One obstacle we have encountered has been the inconsistent methodologies used by different research groups to validate a given device.

This lack of standardization leads to conflicting results and makes it difficult to draw definitive conclusions about device accuracy. A classic example from our research: one study may assess heart rate accuracy during high-intensity interval training, while another focuses on sedentary activities, leading to discrepancies that cannot be easily reconciled.

Other issues include varying sample sizes, participant demographics, and experimental conditions—all of which add complexity to the interpretation of our findings.

What does this mean for me?

Perhaps most importantly, the rapid pace at which new wearable devices are being released exacerbates these problems. Because most companies have an annual release cycle, we and other researchers find it difficult to keep up. The timeline for planning a study, obtaining ethics approval, recruiting and testing participants, analyzing results, and publishing can often exceed 12 months.

By the time the study is published, the device being studied will likely be outdated, replaced by a newer model with potentially different specifications and performance parameters. This is supported by our finding that fewer than 5% of consumer wearables released to date have been validated for the range of physiological signals they claim to measure.

What do our results mean for you? As wearable technology continues to permeate various aspects of health and lifestyle, it’s important to approach manufacturer claims with a healthy dose of skepticism. Gaps in research, inconsistent methodologies, and the rapid pace of new device releases underscore the need for a more formalized and standardized approach to device validation.

The goal would be to foster collaborative synergies between formal certification bodies, research consortiums, popular media influencers and industry so that we can increase the depth and breadth of wearable technology assessment.

Efforts are already underway to create a collaborative network that can foster a richer, multi-faceted dialogue that resonates with a broad range of stakeholders—ensuring that wearable devices are not just innovative gadgets, but also reliable tools for health and well-being.

This article is reprinted from The Conversation under a Creative Commons license. Read the original article.