Today the Washington Post published its investigation of the Q-Collar, a device marketed to “protect the brain” from impacts. My companion essay in The BMJ dives deeper into the device’s evolving history and why I have long been skeptical. This post focuses on the research-integrity side of the story—how the data itself undermines the narrative.
What FOIA Showed—and Why I’m Focusing Elsewhere Here
Both the WaPo and BMJ pieces draw on documents obtained through the Freedom of Information Act (FOIA). Although the device is often touted as “proven safe and effective,” multiple FDA reviewers did not find convincing evidence of effectiveness. The statistical reviewer recommended rejecting the application; the clinical reviewer stated the imaging data fell within measurement error. The authorization ultimately leaned on fine-print labeling to acknowledge uncertainty.
If you want that regulatory backstory, read the WaPo and BMJ articles. Here, I’m going to focus on something more elemental: the integrity of the published data.
If you want insight into why the MRI data are unconvincing, including the FDA reviewer comments, please check out my post in my Human Limits sports science newsletter.
A Sleuth Finds Suspicious Numbers
The catalyst for what follows is Mu Yang, PhD (Columbia University), a neuroscientist and research-integrity expert whose forensic work has preceded hundreds of retractions. She was instrumental in exposing a complex Alzheimer’s research fraud that distorted the field for years.
“Seeing Double” in the Numbers
Mu examined a 2018 paper following 48 female high-school soccer players across a season—27 wearing the Q-Collar, 21 not. In a table reporting neurocognitive outcomes, she noticed something odd: lots of identical numbers.
For the 0-back test, the medians were 0.988 for both groups at both time points (pre-season and post-season).
For the 2-back test, the medians were 0.976—again identical across groups and time points.
Four identical medians across different groups and months-apart time points is… unusual. Suspicious, even. See below:
She set it aside—then later that day opened another paper. This one followed 18 SWAT team members who wore the Q-Collar and were tested before and after a single blast-exposure training session. The table in this paper looked… identical to the soccer table. Check out the image below:
Could this be a simple copy-and-paste mistake? Highly unlikely.
The numbers are identical, yet elements of the tables differ (headers read “pre-training/post-training” vs. “pre-season/post-season”).
Sample sizes differ between papers.
The soccer study includes an extra abbreviation (A′ for accuracy) that’s absent from the SWAT table.
Footers and labels are tailored to each paper.
I’m letting the non-italicized vs. italicized “t” and “p” in the upper right slide here, assuming those formatting inconsistencies could have be done by the journal editors
For this to be a mere paste error, the authors would have had to copy the soccer table, manually edit multiple labels and counts, remove A′ in one version, update footers—and somehow not update the actual data. That stretches plausibility.
We (Mu, I, and a team of nearly a dozen collaborators who reviewed the findings and agreed they were problematic) reported this and several other anomalies to the Journal of Neurotrauma, which had published both papers. Several months after our initial contact, the authors issued a correction for this study—along with two others where we identified data irregularities. Some of the concerns we raised remain unresolved, and the journal has since issued a formal Expression of Concern covering six studies, including these two.
The Correction… Still Has Impossible Math
The authors wrote that they had “inadvertently provided a wrong version of Table 4” and supplied a replacement. See below:
But in the updated table, the 0-back collar group lists t = 1.43 with p = 0.62. Those are not mathematically compatible. A t-statistic around 1.43 should yield p ≈ 0.20, which—tellingly—appears for a nearby entry: t = 1.42, p = 0.20 (2-back, non-collar group). There’s only one 1.43 and only one 0.62 on the page, so this isn’t a simple transposition from a duplicate row.
Version one had a duplicated table. Version two still can’t reconcile t and p.
Do we grant another do-over?
The Trust Problem (Not the “Significance” Problem)
Authors often assure readers that such errors “do not alter the conclusions.” Maybe none of these particular rows were statistically significant anyway. But that response misses the point.
Scientific publishing runs on trust. When a research group can’t deliver a single clean table on the second attempt—after months to double check and with 15 co-authors—why should we be confident they flawlessly managed the underlying data pipeline, analyses, and the rest of the paper’s tables and figures? What about the errors we can’t see?
Likewise here, intent isn’t the hinge. Maybe these really are true mistakes. But the cascade of visible errors—identical medians across unrelated cohorts, then an incoherent t/p pairing in the correction—makes the narrative unreliable, even if it’s not intentional deception.
But this isn’t an isolated misstep. The same authors have issued corrections on two additional papers, and six Q-Collar studies are now formally under investigation by the journal. At some point, these can no longer be dismissed as isolated oversights; they reflect a systemic problem in the research process. That doesn’t necessarily mean anyone set out to deceive—but it does mean there are a lot of mistakes. And in science, repeated errors—even innocent ones—erode confidence just as surely as misconduct.
Let’s also throw in outcome switching and biased interpretation and arithmetic errors and the whole story of “brain protection” becomes less and less convincing. The problem isn’t just a bad table—it’s a body of work that keeps asking for more trust than it has earned.
My view is simple: when reliability collapses, trust follows. And once trust is gone, the conclusions, no matter how statistically “significant” or “insignificant,” lose their meaning.
Why This Matters Beyond One Device
This isn’t about catching a typo. It’s about how small, concrete anomalies reveal fragile foundations. If peer review and post-publication corrections can’t catch or fix basic arithmetic inconsistencies, how confident should we be in more complex claims (like subtle imaging changes interpreted as “brain protection”)?
Errors we can see are usually the tip of the iceberg.
Where Trust Actually Breaks
Every scientist knows that mistakes happen. What matters is how often they happen, how they’re handled, and whether the corrections restore confidence—or deepen doubt. In my opinion, fixes in the Q-collar research raise more questions than they answered.
Trust doesn’t vanish all at once. It erodes quietly, number by number, as small inconsistencies accumulate into a pattern that can’t be waved away. When the same research program produces duplicated tables, impossible statistics, shifting outcomes, and a trail of corrections under investigation, skepticism stops being cynicism—it becomes due diligence.
The point isn’t to assume bad intent. It’s to recognize that science’s authority depends on transparency and reproducibility, not marketing claims or institutional endorsements. Once those foundations crack, the data cease to be evidence—they become a story we can’t verify.
And that’s the moment a study, or an entire body of research, truly loses our trust.
Do you think the duplicated data (with different row and column headers)—and the still-flawed correction—are enough to undermine trust in the entire study? Or would you give the researchers the benefit of the doubt? I’d love to hear how you draw that line.
Peer review works when the reviewer is skeptical of the study. Then their comments serve to strengthen or refine the argument being made and supported. A reviewer who is not a skeptic does not bring a "beginner's mind" to the work, so that shared assumptions, which may be critical, never become conscious.
If you don't pay reviewers, it really is possible for them to miss stuff this obvious.