Skip to main content

Calorie counting leads to bad science and worse gadgets

Calorie counting leads to bad science and worse gadgets

/

The mirage of accuracy can mislead and sabotage instead of motivate

Share this story

If you buy something from a Verge link, Vox Media may earn a commission. See our ethics statement.

Technology has brought unprecedented precision to our daily lives. I can draw a perfect circle with my eyes closed, calculate Pi to the nth degree, split a 15 percent tip seven ways, and accurately measure how many calories are going in and out of my body. Or can I?

A frenzy for quantifying everything about the human body is sweeping the electronics industry — with Apple's HealthKit and Samsung's SAMI being the latest and most prominent efforts — and catalyzing an explosion of sensor-laden wearable devices. But in its hurry toward a more self-aware future, this wave of innovation is also perpetuating one of the big mistakes of the past: counting calories.

As food historian Martin Bruegel explains, the obsession with calories got its start at the turn of the 20th century as part of a broad "scientific eating" movement focused on figuring out ways to help the poor eat well. Over time, that concern moved from the political to the personal, culminating in the 1997 introduction of the Weight Watchers Points program. That globally popular dieting regimen elevated the calculation and tracking of calories into a weird blend of pseudoscience and ersatz religion, asking adherents to follow the numbers with unquestioning faith.

Even Weight Watchers now accepts that calorie counting's a bad idea

That’s where the pre-digital world went wrong. "As the co-author of a book called Why Calories Count, I might be expected to favor calorie counting but I most definitely do not. It cannot be done accurately or even close to accurately," noted NYU nutritionist Marion Nestle tells The Verge. In November 2010, then-president and CEO of Weight Watchers David Kirchhoff conceded that point and admitted that "calorie-counting has become unhelpful" for people trying to lose weight. Citing the myriad variables that go into working out how much energy is in your food and how much of it you can actually digest, his company completely reworked its program.

In spite of a widespread professional unease about getting too specific with calories, that’s exactly what new technology is doing. The problem is that standards for accuracy and reliability of consumer health gadgets are not as strict as those for professional use because, well, they don’t exist. The US Federal Trade Commission investigates and takes action against allegations of false advertising or misleading claims, but that’s about as far as it goes. Pretty much anyone can introduce "the next great health invention," and with the help of crowdfunding platforms like Indiegogo, many are trying.

Opportunity meets opportunism

A million dollars. That’s how much faith the internet has put in the promise of Healbe GoBe, an Indiegogo project that has raised as many red flags as funding pledges. This wrist-worn device claims to combine pressure and hydration sensors with an accelerometer to accurately quantify both the calories you’ve taken in and the calories you’ve burned. The key metric being measured by the GoBe wearable is said to be glucose concentration in the blood, though Healbe’s own literature on the subject admits it "doesn’t meet medical accuracy requirements." If it did, it’d be a revolution for diabetics, but the fact it doesn’t isn’t stopping the company from promoting it to people desperate for a new method to lose weight. Beyond calories, Healbe’s claims grow increasingly outlandish: it’ll also tell you the macronutrient (fats, carbohydrates, and protein) composition of the last meal you ate and monitor your stress levels.

Healbe560

You can train for years to become a nutritionist or pay $2.99 for an app that will guess the calories for you

The $2.99 Meal Snap app follows a similar path by offering to calculate the caloric content of your food through "pure magic." You take a photo of your plate, upload it, and the app returns an estimate of how many calories sit in front of you. The entire mysterious process appears to actually be a case of manual photo analysis that matches the results to the food database of Daily Burn, the app’s maker. Meal Snap is not without merit, as it helps users keep a visual food diary over time, but its results have been shown to fluctuate wildly.

The calorie content of anything you eat will vary, sometimes dramatically, according to how it’s cooked, plus a proportion of a meal’s energy is always lost to bacteria inside the gut and the digestive process itself. All this complexity is lost when you have a person being paid a few cents per photo identifying it as "chicken with small bits."

This creates a dissonance between perceived and real accuracy. When you’re told that "that piece of bread has 113.95 calories," as in the demo video for The Situ Smart Food Nutrition Scale, the extra decimal places suggest precision. But all you’re doing is weighing each ingredient in your meal and then assigning it a generic caloric value for that typical food. You think you’re adding 100 calories of ham and 32 — no, let’s do two slices — 69 calories in the form of cheese, but those are estimates, a guide rather than a hard rule. Like the GoBe, the Situ has met with crowdfunding success, this time through Kickstarter, but like Meal Snap, it seems to be focusing on the wrong thing.

The lack of precision in self-reported food consumption is "legendary"

The common thread that runs through calorie-counting apps and gadgets is that we, as humans, are not very good at it. Professor Nestle describes the lack of precision in self-reported food consumption as "legendary," citing an average underestimation of 30 percent. But the same pitfalls that sabotage pen-and-paper calorie tracking also affect the digital kind: you’re still just as likely to neglect documenting late-night snacks as you ever were. In the absence of an automated solution like the GoBe that actually works, technology’s just fancying up an inherently faulty process of self-quantification.

Hard numbers don’t matter as much as comparable ones

Measuring the expenditure of energy — the other side of the caloric equation — has produced a wider range of automatic devices, but not much more accuracy. Most pedometers struggle to keep an accurate step count, never mind calculating the exact energy consumption of a given day’s exertions. Dr. Alice Ammerman from the University of North Carolina's nutrition department says that "probably the most useful health-promoting information from these devices is to show that it takes a lot of physical activity to ‘burn off’ a candy bar or soda, or other high-calorie food."

It'd be okay for every device to be wrong if they were all wrong by the same amount

Another significant advantage of fitness trackers like the Nike FuelBand is that they allow for comparison: you can track your personal development over time, compare times with others online, or just measure yourself against a general standard. That doesn’t strictly need to be accurate, it just has to be consistent. And it is, provided you only ever use Nike products, but the legion of other fitness monitors on the market don’t count Fuel points and don’t adhere to any other universal standard for measurement. Apple’s newly introduced HealthKit proposes to rectify one of these issues by centralizing data from all wearable devices into one hub, but it doesn’t yet offer a solution for the problem of inconsistency. A recent test by Quartz found an average disparity of over 500 calories per day between some activity trackers.

In the absence of standards, people resort to calories as the common denominator for cross-comparisons, using that seemingly scientific measurement in a way that’s far from scientifically rigorous. You can’t balance your diet properly with a Situ scale and a Samsung Gear Fit — even though both purport to track calories — because they use different methods and produce different errors in their final numbers.

Scale-aria_1

Dr. Barry Popkin, a colleague of Professor Ammerman’s at UNC and the author of The World is Fat, agrees with her and Marion Nestle in warning against the dangers of even small errors of estimation when relying on calories to guide lifestyle choices. "The issue is in how do people use them," he says, "and do the Fitbits, etc., overestimate movement." A system that consistently miscalculates energy use risks being falsely reassuring or forcing the user into an unnecessarily restrictive diet. You’re better off, argues Nestle, buying a scale to weigh yourself and judging whether to eat more or less by looking at whether you’re gaining or losing weight. It’s a breathtakingly basic approach, but until the technology to achieve true "scientific eating" matures, it’s the best option on the table.

Samsung and Apple could bring order to the data chaos

Aside from issues of accuracy and consistency, the crucial missing element from wearable health gadgets today is user education. Knowing the limitations of biotelemetry and how to interpret it properly is fundamental to making good use of that information. Samsung’s new Simband prototype is essentially a portable clinic, tracking every imaginable health parameter, but its most useful promise is the ability to share its findings with your doctor. Apple’s HealthKit similarly aims to act as a digital library of self-quantification data that’s accessible to health care providers. Dr. John Noseworthy, CEO of the Mayo Clinic, says it "will revolutionize how the health industry interacts with people."

Both Samsung and Apple’s efforts have the potential to be transformative. Being the two biggest companies in personal electronics, they can set real standards for accuracy and reporting that others would be willing to follow. Instead of do-it-yourself calorie comparisons, people would have access to professional clinical assessments. But the road to making that vision a reality is long and we’re only just getting started. Along the way, we'll do well to avoid junk science as well as junk food.