A widely quoted digital physiotherapy result — “72% of the program participants had clinically meaningful pain reduction at 12 months” — rests on the people who were still answering surveys a year later. The paper says how many that was in aggregate. The press release does not, and it is the press release that reaches the employer deciding whether to buy. Below, your own browser pulls the paper’s abstract from two independent public APIs and does the arithmetic in front of you.
“On average, Hinge Health program participants reported 51% pain reduction sustained over 12 months. 72% of the program participants had clinically meaningful pain reduction at 12 months.” hingehealth.com → press release, 23 March 2022
“Researchers … compared Hinge Health’s digital MSK program participants with nonparticipants (n = 2570).” same press release, quoting the paper’s abstract
Read as an employer reads it, that is: out of about 2,570 people, roughly seven in ten got meaningfully better. Both halves of that reading are wrong, and the paper itself says so.
The study is Clinical outcomes one year after a digital musculoskeletal (MSK) program,
BMC Musculoskeletal Disorders 2022, 10.1186/s12891-022-05188-x — open access,
so every sentence below can be checked in full.
“Overall, we achieved a 70% response rate at 3 months (2277/3265), a 52% response rate at 6 months (1422/2732), and a 50% response rate at 12 months (859/1710).”Results → Flowchart
“The primary analysis employed complete case analysis, i.e., excluded missing values.”Methods → Statistical methods
“Table 2 Study sample characteristics at baseline Comparison (N = 1650) Intervention (N = 2720) Total (N = 4370)”Results → Table 2
So the twelve-month percentage is computed over 859 people who answered, out of a baseline sample of 4,370. Everyone who stopped answering — whether because they got better, got worse, or left the employer — is not in the denominator at all. That is not a hidden practice: the authors state it plainly and even measure what it costs.
| Odds of a meaningful pain improvement vs. non-participants |
Primary analysis (complete case) | Sensitivity analysis (missing data imputed) |
|---|---|---|
| at 3 months | OR 1.97 | OR 1.37 |
| at 6 months | OR 1.44 | OR 0.96 |
| at 12 months | OR 2.06 | OR 1.38 |
Those are the paper’s own two columns, adjusted models, quoted from its Results. When the people who stopped replying are restored by multiple imputation, the twelve-month odds ratio falls from 2.06 to 1.38, and the six-month figure crosses to 0.96 — below one. Neither imputed number appears in the press release. Neither does the comparison group: the paper reports the gap between participants and non-participants at 12 months as 16.0 percentage points, and remarks that “over half of the intervention and nonparticipant groups experienced meaningful pain improvements”. People who registered and never started the program also got better. The number that belongs to the program is the gap, not the 72.
The abstract says n = 2570. The paper’s own components are printed in the Methods and Table 2, and they add up to a different figure. Press the button: your browser fetches the abstract from Crossref and from Europe PMC — two independent registries, neither of them mine — and then sums the cohorts.
On another page a published rate turns out to be arithmetically unreachable over the population named beside it — a closed contradiction. This one is the quieter version: nothing here contradicts anything. Every figure is real, the paper is candid, the statistics are competent. The loss happens in transmission. Between the Results section and the press release, three qualifiers fall off — among those who responded, complete case, versus a comparison group that also improved — and what arrives at the buyer is a bare 72% attached to the word “participants”.
Note also the precision. “72%” carries no decimal, and a whole-number percentage is compatible with almost any denominator; a single decimal place would already narrow it to a handful. Dropping the decimal is not deception, but it does remove the last trace of the count — which is why the check is worth running on every rate before it is quoted back to you.
Not “is this study any good” — it is better than most. Ask instead, of any outcome figure that money is attached to:
How many people are in the denominator of that percentage, and what happened to everyone who was enrolled but is not in it?
If the answer is a number, the figure can be audited. If the answer is a category — “participants”, “members”, “engaged users” — the figure has already lost the only thing that made it checkable.