5 Tips to Better Understand Research: A Beginners Guide

Zachary Walston

“Scientific knowledge is a body of statements of varying degrees of uncertainty/certainty — some most unsure, some nearly sure, none absolutely certain.” — Richard Feynman

Does that make you uncomfortable? We never have a final answer. Ever.

Does it have to be as Feynman describes the relationship between scientists and the general population?

“One of the important reasons to increase the contact of scientists with the rest of society is to explain and to kind of wake them up to the permanent attribution of the cleverness of the mind that comes from not having information always in a for which is interesting.” — Richard Feynman

I think contact between scientists and lay individuals is very valuable, but if we rely on scientists to be the only source of translating research, misinformation will continue to infiltrate and infect social and popular media.

Curiosity should be fostered in all people. For curiosity to thrive, doubt must be ever-present. That is uncomfortable. It also means the work never ends.

This brief guide is meant to arm you. I want to help you identify misinformation, question “conclusions” drawn by social media influencers and reporters. This guide will help you read and understand research articles. Here are five things you need to know when reading research.

1. The Power of the Study

Are there enough study participants to draw a firm conclusion? For example, if you are trying to figure out America’s favorite football team, you wouldn’t draw a conclusion after polling 50 people.

In his book, Thinking, Fast and Slow, Daniel Kahneman provides a more eye-opening influence of underpowered studies.

“A recent study on the prevalence of kidney cancer in 3,141 counties across the US found that counties with the lowest rates are mostly rural, sparsely populated, and located in traditionally Republican states in the Midwest, the South, and the West.”

Take a moment and consider the reasons. Now let’s look at the counties with the highest rates of kidney cancer:

“They are mostly rural, sparsely populated, and located in traditionally Republican states in the Midwest, the South, and the West.”

This is not a typo. The descriptions are the exact same. Why? The answer lies in the law of small numbers. The key is ‘sparsely populated.’ Low numbers result in high variability. The counties with a small number of residents have the greatest variability in rates.

2. Who is being studied?

Every person is unique and every person has a unique life. A study containing 1000 people with chronic low back pain is not studying 1000 identical people with identical lives. The study may look at the effects of an exercise protocol, but does it control for the other activities of someone's life (diet, sleep, stress)? In what ways do the people differ (social support from friends and family, desire to participate, history of exercise)? What other characteristics aren’t accounted for (fear of activity, body mass index, pain or injury in other body regions, occupation). All of these factors can influence the results of the treatment.

We need to know who the study applies to. This is why we can rarely draw conclusions from a single study.

3. What was actually done? What wasn’t done?

The abstract — which is the short summaries you find on the PubMed page and the beginning of a research article — rarely has the details of the intervention. This is an issue.

A research study may conclude “exercise is beneficial for neck pain.” Great! But, what kind of exercise?

You need to know the type of exercise, the duration, the intensity, etc. True, the body of research overwhelmingly favors exercise for nearly any pain experience, however, optimal results can be achieved with specific approaches. It is important for us to know the details of what was completed.

4. Do the results matter?

“Just because we can detect an effect does not mean it matters” — The Power of Mathematical Thinking by Jordan Elleburg

If a medication lowered your low back pain 0.5/10 points for 1 hour, would that be meaningful to you? What if insurance didn’t cover it and the medication had several side effects? Probably not. Yet, here is a sample headline:

“New Drug Shows Promising Results for Reducing Low Back Pain.”

Slightly misleading, no?

The other issue is cherry-picking data. For example, several large prospective cohorts have concluded eating red meat causes increased mortality risk. Unlike in smoking studies, the results are not consistent. In addition, the meat-eating people included in the study also have the following characteristics compared to the vegetarians:

  • Decreased exercise frequency and duration
  • Increased BMI
  • Increased smoking frequency
  • Increased prevalence of diabetes
  • Increased total caloric intake
  • Increased alcohol intake

This leads to a couple of questions. Is it possible that elevated BMI, lack of exercise, excessive calorie intake, or a combination caused the increased mortality rates and not the red meat? It is very possible the red meat caused the development of those conditions; however, an association is not a cause and effect relationship.

5. The Type of Study

All studies are not created equal. The above pyramid is a quick guide to the research hierarchy. Why are randomized control trials at the top? Without a control to compare against, we cannot know whether an effect occurred.

For example, if I wake up with back pain but go to bed without any, what caused the change? Was in the massage I received during my lunch break, the afternoon workout, the recent switch to a plant-based diet, the good news I received at work, or the relaxing movie I watched in the evening? Maybe it was a combination of all four. I cannot know for certain without a time machine and the ability to compare days without one of those qualities.

A trial would compare massage to no treatment to see if the message has an effect. Then layer in the size, the population, and the details of the massage. Then determine if the effect is large enough to matter, if it lasts, and if it is better than other options. Lastly, has this study been reproduced or was it an anomaly?

Now what?

One of the big challenges is finding research as many are behind paywalls. There are a couple of options.

  1. Find a credible source of how to provide objective summaries of research. Here are a few sources:
  2. Read the abstract but understand the limitations. Key information is often left out
  3. Request the full text directly from authors on Research Gate or social media

This takes work. There are more concepts I did not cover that I encourage you to learn about: blinding, p-hacking, bias, and statistics basics. There is no quick solution to reading research, but if we are to better understand information, it is a must.

Comments / 0

Published by

I am a physical therapist, researcher, and educator whose mission is to challenge health misinformation. You will find articles about health, fitness, medical care, psychology, and professional development on my site. As the husband of a real estate agent, you will also find real estate and housing tips.

Atlanta, GA

More from Zachary Walston

Comments / 0