P-Values and Causality – What You Should Really Look for in Studies
Statistically Significant – So, Important?
You hear it all the time: “This result was significant!” It sounds big, almost like a seal of approval. But what does it actually mean?
Statistical significance ≠ clinical relevance
Statistical significance is often misunderstood. In science, it’s usually expressed by the “p-value.” In medicine, thresholds of 0.01 (1%) or 0.05 (5%) are most common. Put simply, the p-value indicates how likely it is that an observed result arose purely by chance, assuming there is in fact no “real” difference between the groups being studied.
Sounds abstract. Let’s make it concrete: Imagine you run a study testing whether Drug A works better than a placebo. Two groups are treated: one gets the drug, the other gets a placebo. You see a difference: Drug A performs better.
Now you ask: Is that difference just random—or is it really the drug?
A “p = 0.05” means there’s a 5% probability of observing the same or greater difference in your study if in reality no difference exists between the groups. So, the p-value suggests the effect of Drug A is unlikely to be random—but it does not prove that the result is clinically relevant.
Statistics are important, but they are not everything. A drug might slightly improve a lab value, yielding a statistically significant result, yet provide no meaningful benefit to patients. A result just under 0.05 is not “true,” and one just above is not “false.” Clinical relevance is about real improvements: fewer illnesses, fewer symptoms, better quality of life. Those can’t be reduced to a single number.
The idea that 0.049 is “valid” while 0.051 is “irrelevant” highlights how arbitrary the cut-off really is. In reality, we’re dealing with a continuum of probabilities, not a black-and-white distinction. The p-value is a statistical clue, not a clinical verdict.
Correlation Is Not Causation
Equally important is the distinction between correlation and causation. You often see headlines like: “People who do X have a higher risk of Y.” That may sound like a direct link, maybe even cause and effect. But beware: just because two things occur together doesn’t mean one causes the other. Both might be linked to a third factor. Or it might just be coincidence.
Good studies—especially randomized controlled trials (RCTs)—aim to filter out these confounders and separate correlation from causation.
RCTs vs. Real-World Evidence – Controlled Doesn’t Always Mean Practical
Still, RCTs are not the “holy grail” or an automatic guarantee of truth. If you want to know how well a therapy works under ideal conditions, you look at RCTs or at meta-analyses that combine them. These are the gold standard of evidence, because randomization and controlled conditions minimize bias.
But as rigorous as they are, RCTs often show what works in the “lab,” not what works in everyday practice.
That’s where real-world studies come in. They examine how treatments perform in actual healthcare settings—among patients with comorbidities, of different ages, and in diverse life situations. These data are less tightly controlled, but much more practical.
Both study types matter:
RCTs: high internal validity, strong causal conclusions—but less reflective of daily reality.
Real-world studies: less controlled, but closer to real life and often more relevant for broad populations.
Excitingly, hybrid approaches like pragmatic trials or effectiveness studies are emerging. They aim to combine both: controlled methods with real-world questions.
The Bottom Line
Good medical studies provide valuable insights. But:
A p-value is not a verdict.
A correlation is not causation.
An improved lab value does not automatically equal a better life.
What matters most is context—and context is usually more complex than headlines or marketing slogans suggest.
If you want to better understand studies—or to know what a diagnosis, lab value, or risk figure really means for you—you need more than data or PubMed articles. You need interpretation by a physician.
At YEARS, we take time for exactly that. You don’t just get numbers—you get medical context. We discuss your results with you, explain them in plain language, and put them into perspective for your life. Professional, clear, personal.
Want clarity? Ask us. www.years.co
References (selection):
Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research. NEJM. 2000;342(25):1887–1892.
Frieden TR. Evidence for Health Decision Making — Beyond Randomized, Controlled Trials. NEJM. 2017;377:465–475.
Ioannidis JPA. Why Most Clinical Research Is Not Useful. PLOS Medicine. 2016;13(6):e1002049.
Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician. 2016;70(2):129–133.
Sharma H. Statistical significance or clinical significance? A researcher’s dilemma for appropriate interpretation of research results. Saudi Journal of Anaesthesia. 2021;15(4):431–434.