Hey everyone, it's Mikki here. You're listening to Mini-Mikkipedia on a Monday.

Before I go on, if you like these episodes, or just Mikkipedia in general, can you subscribe

on your favourite podcast listening platform, share it with a friend, because that then

increases the visibility of Mikkipedia in amongst literally thousands of other podcasts out there.

And if you feel so inclined, give me a five-star review. That would be amazing.

All right, what are we talking about this week on Mini-Mikkipedia? I got sent an email from one

of my clients, and she had a link to a very flash Withings scale. And I'm like, oh, this is

interesting. And oh my goodness, it's 819 New Zealand dollars. JB high five. And it is a,

essentially, it's a bioelectrical impedance scale, but apparently it's like 99% accurate.

And I'm like, really? This sounds super interesting.

And basically, my client was like, Mikki, is this worth the money? Because she's in the market

for something that's actually worth its salt. And she says, I know that my usual scales just track

kilograms, but I get a DEXA every year, but I'm still curious as to whether anything on the market

right now is actually going to give me more information than that. And so that 99% as

accurate as a DEXA as marketed was like, oh, that's super interesting. And with that price tag, I'm like,

really? So I understand why that is appealing. Clearly, if you've ever paid for a DEXA scan,

you know, it's not cheap. Well, you know, a few hundred bucks, at least you have to book it,

you have to drive there, and you get one number on one day, whereas a scale in your bathroom,

if that gave you that same information every morning for the price of like three scans or

whatever, pretty appealing. So I went to read the research that the company links to. So I want to

walk through what I found. And I'm like, oh, this is really, really, really, really, really, really,

really, really, really, really, really, really, really, really, really, really, really, really,

because the problem here isn't really about just one scale. It's about a statistical issue that

shows up constantly in health tech marketing. And it's worth you knowing if you're at all

interested in this. And for what it's worth, people always ask me about these BIA machines.

So the claim that the Withings scales was 99% as accurate as a DEXA rests on this validation

document that the company had produced. There were 80 people, each of them had one DEXA

scan and three measurements on the scale. And the headline result was a correlation of 0.99 for

total fat mass. Now 0.99 clearly is a very high correlation. And that part is clearly legit.

But what isn't true is that correlation of 0.99 means that the scale is 99% accurate.

A correlation tells you whether two things move together in the same direction. It tells you that

people who have more fat mass on DEXA also have more fat mass on the scale.

That's it. That is all that it tells you. So imagine three people. DEXA says they have 10,

20, and 30 kilos of fat. The scale says 5, 10, and 15 kilos. Every single reading is wrong by one

half. The scale has failed completely. But the correlation between these two sets of numbers

is a perfect 1.0. It's exactly the same, yet doesn't really give you great data. And what you

want when you're comparing two measurement methods is agreement, not correlation. You want

to know the mean difference between the two. And you want the limits of agreement, which tell you

how far off an individual person's reading is likely to be. And that's something called a

bland Altman analysis. And it's been the standard way to do this since the mid-1980s. It actually

isn't obscure. I mean, I'm just going to say right up front, I know next to nothing about this. I

believe I used a bland Altman in one of my papers. I don't know if you've seen it. I don't know if

it's actually from my thesis, but I cannot recall. But it's certainly not a niche statistical

preference. It's the thing that you do. And they didn't do it. So I think this is always something

interesting when you look at research and you know, or you learn that there is a particular

statistical analysis that really suits a question. And it's completely ignored by the people doing

the analysis. So that's worth noting. What I will say, however, I don't think the technology is

rubbish at all, right? With that withings.

So I think that's a good thing. And I think that's a good thing. And I think that's a good thing.

So I think that's a good thing. And I think that's a good thing. And I think that's a good thing.

This particular one uses eight electrodes, including a handheld bar, and it does run multiple

frequencies. So that means the current passes through your arms, your trunk, and your legs,

rather than just up one leg and down the other, which is what a cheap bathroom scale that I got

at home does. So it is actually better hardware. But the evidence is a separate question from the

hardware. So what I couldn't find anywhere in that material was the mean difference from the

DEXA. The limits of a green screen are not as good as the green screen. So I think that's a good thing.

That individual prediction error, the age and sex breakdown of the participants,

the body fat range, whether hydration and fasting were standardized, as I've talked about with

Professor Grant Tinsley on this podcast, who is a body composition expert, whether the validation

sample was independent of the data used to build the prediction algorithm in the first place.

And that does actually matter. And I couldn't actually find any peer reviewed publication

of that study. Something else that's worth

sort of chatting about is that the results are presented as though all four measures were

validated against DEXA, but total body water, which is something else you can glean from

these scales, was compared against an entirely different device. And I mean, of course, that

makes sense because DEXA doesn't measure total body water. But it's worth noting that because

it does tell you something about how carefully that document was put together. Basically,

it's all marketing speak, and they're going to do as much as they can,

to convince you of the accuracy and the utility of that scale. And then there was a line in it that

says in the marketing that scale does not merely provide an estimate. I mean, it absolutely provides

an estimate. That's all it's providing. This is what that technology does. The device measures

your weight, and it measures electrical impedance. Everything else, your fat mass, your muscle mass,

your visceral fat, your metabolic rate is calculated from those two inputs,

plus your age, your sex, your height. It is run through this propriety algorithm. And the

company's own technical explanation of the method tells you all of this, so that the entire thing

is an absolute estimate. And then, you know, it is worth noting that there is good independent

research on how accurate bioelectrical impedance actually is. And in January this year, there was

a systematic review published in the Journal of Functional Morphology, which is a journal that

looked at bio-impedance devices compared against a four-compartment model, which is stronger

reference than a DEXA alone, because it accounts for body volume, total body water, bone mineral,

and body mass separately. There were 12 studies, and across those studies, the mean bias for body

fat percentage ranged from about three and a half percentage points, too low, to four and a half

percentage points, too high. But the number I want you to look at is the number of studies that

really think about is the limits of agreement, which typically spanned 15 to 20 percentage points.

That's 15 to 20 percent. Wow. And the authors concluded that the BIA estimates of body fat

percentage and fat-free mass were not equivalent to the reference model. And I mean, that does

sound damning, and at the individual level it is, but there is a second finding that does make me

think, you know, that may change how you should use these devices.

So in a 2023 study in the British Journal of Nutrition, Seidler and colleagues put 73 adults

through 15 different BIA devices and compared them all against a four-compartment model. 37 of those

people came back 12 to 16 weeks later, so the researchers could look at whether the devices

detected change accurately. The reliability was excellent. Precision error across the devices

ranged from zero to about half a percentage point, which means that these

devices are very consistent with themselves. So that is a key distinction. Reliable does not

mean accurate. A device can be reliably wrong. If your scale reads three percentage points

high every single time, your absolute number is wrong, but the direction and the size of the change

might still be informative. The longitudinal results in that study I just mentioned were

more mixed though. Constant error for detecting change ranged from about half a percentage point

to just over one percentage point too high. So really it's telling you that some devices

tracked change quite reasonably, but others really didn't. And in June of this year,

there was a study published in the Journal of Primary Care and Community Health

looking at the InBody H20N, which is a consumer-grade portable device

against DEXA in 55 adults with obesity. Their average age was 62 and two-thirds of them were

women.

Participants fasted for eight hours, stayed hydrated, and avoided exercise and alcohol for

24 hours beforehand, which is, you know, sort of gold standard practice, standardizing that stuff.

For body fat percentage, the agreement with DEXA was excellent by the usual statistical standards.

And they reported actual numbers. Mean difference of 1.64 percentage points,

limits of agreement, minus 3.4 to plus 6.7 percentage points. So on average across

the group, it's quite good. For an individual standing on it, the true value could plausibly be

three and a half points below or nearly seven points above what the screen actually says.

And for skeletal muscle mass, it was worse actually. The device overestimated by an average

of 5.66 kilos. That's pretty huge compared to the DEXA with the limits of agreement spanning about

10 kilos. So I mean, that's what proper reporting looks like. It's way less impressive than the

90-day-old. But it's a good thing. It's a good thing. It's a good thing. It's a good thing. It's a good thing.

But it's infinitely more useful because now you know how much you can actually trust that number.

So two other things about that study. It was the first to compare that particular device,

the InBody H2O-N against DEXA. And the authors did say that that longitudinal agreement,

so whether it actually tracks change accurately over time, that hasn't been established.

So that is a critical next research question.

They do actually sell that machine, the H2O-N. As I said, it's a direct-to-consumer

device. But InBody actually have a range of different devices, like the Dial H20,

and then it's an H20B, and an H30, and an H40 in the same family. So these different models

haven't been tested the way that the H, I called it the H20N, it's probably the H20B,

has. So just note that, I think. And then if you look at how much it costs,

like I started off this episode by telling you that the Wything scale was about 819 New Zealand

dollars. The H20N InBody is 529 Australian dollars, but I couldn't find a New Zealand

retailer actually stocking it. So it's cheaper than $800, but it's still not cheap.

So here's something I just want you to bear in mind, regardless of what scale you,

have. These are the metrics that generate pretty much a lot of attention, but are not worth paying

attention to. The first one is visceral fat. On most of these devices, this is a propriety index

on an arbitrary scale, meaning it's basically made up by the manufacturers, calculated from

impedance, plus your age, sex, height, and weight. It is not a measurement of visceral adipose tissue

in the way that a CT or MRI would give you. So don't treat a number going from nine to eight

like it's something worth celebrating.

Because really, it doesn't tell you anything. It's almost made up. Bone mass. This is not bone

mineral density. Bioimpedance cannot tell you anything meaningful about your skeletal health

or your osteoporosis risk. So if this is a concern of yours, then a DEXA scan is absolutely your best

bet, because that is gold standard. Basal metabolic rate. This is calculated from an equation. It is

not measured. Measuring it requires indirect colometry, which involves a mask and a metabolic

card. And, you know, I don't want to go into too much detail about this, but I do want to

know what you think about this. And I'm going to go into a little bit more detail about this.

It basically will be derived, again, from an algorithm. And depending on the population that

they derived it from, it's as accurate or otherwise as you would expect. So I just would

not hang my hat on it. And body water. This is better informed on that multi-frequency device,

like those eight different levers, than a cheap one. But it's still derived from an electrical

model of your body fluids. So do I think you should throw your scale out? I mean, clearly not.

I think you should throw it out. I think you should throw it out. I think you should throw it out.

You should use it for what it's good at. So here's what I'd do. You want to standardize

everything you can control. Same time of day, first thing in the morning, after you've been

to the toilet, before food, fluid, or training. Hydration, glycogen, and even where you are in

your cycle will move these numbers around. And they'll move them around more than a real change

in body composition will over a short window. So there's a lot of noise. So you want to measure

often. Three or four times a week is fine. Then average it across a week.

Look at four-week trends, not daily readings. Your body fat estimate drops from 27.2 to 26.6.

That's probably noise. But if it drifts consistently downward across two months of weekly averages,

that's a signal. So that's a good thing. And then you want to triangulate. Don't just look at what

your in-body tells you. Look at body weight, waist circumference, how your clothes fit,

what's happening with your training numbers, photos if you find them useful. One measurement

is a data point, but several measurements pointing in the same way. That's information.

And then change that sentence you say to yourself of, you know, my body fat is 23.7%.

Instead, think, my body fat estimate has trended down over the last 10 weeks. So that is much more

accurate than that first sentence that I described. What I do want to be clear on is that, I mean,

the Wything scale is as good as the in-body H20N. You know, like,

I went in and read the in-body's own marketing after I read their study. And on their product

page, they claim their technology has shown over 98% agreement with DEXA, which is pretty much

exactly what the Wything scale said. So the entire category market correlation is accuracy because

correlation coefficients are, they're pretty flattering, aren't they? But agreement statistics

are not. And those are the ones that we want to be looking at. The only reason we know anything

about that.

In-body scale is that a research group in Alabama went and did the work themselves,

published the limits of agreement and told us plainly that the device overestimates muscle

by five and a half kilos. Obviously, a manufacturer is never going to put that on the box, right?

And I guess, really, when it comes down to it, the point of this episode might not even be about

those scales at all, right? When someone tells you a number is 99% accurate, ask what they mean.

Ask what it was compared against, who paid for the comparison, and how far off an

individual reading can be. Because those three questions will get you through most of this

health tech that you'll be sold on in the coming years. And isn't there a lot of health tech these

days? So I will say that, you know, a well-built consumer bioimpedance device can definitely be

useful for tracking direction over months, but it's certainly not giving you 99% accuracy. But

as long as you're aware about that, you triangulate that data, then I think you're good. If that is,

you actually want to pay that $800 that these are selling for. Anyway, I'd love to hear whether

you've got one of these machines, and you find it useful, or if you found this useful,

hit me up. I'm over on Instagram, Threads and X, at Mikki Williden. I'm at Facebook,

at Mikki Williden Nutrition, or head to my website, mikkiwilliden.com,

and scroll right down to the bottom, pop your name in the box to jump on my email list.

All right, guys, you have the best week. See you later.

Bye-bye.