Demystifying Scientific Studies: A Guide to Reading and Interpreting Medical Research

The Scientific Process and Types of Studies

Before a scientific study even begins, researchers start by forming a hypothesis about a particular relationship between variables. From there, they design experiments to test that hypothesis, get approval from ethics committees, determine the number of human or animal subjects needed, outline key outcomes to measure, plan statistical analyses, and pre-register the protocol.

Studies generally fall into three main categories:

 
• Observational studies – These studies observe individuals over time but do not deliberately intervene. Types include individual case reports, case series, retrospective cohort studies looking backwards in time, and prospective cohort studies following subjects forwards.
• Experimental studies – Researchers directly intervene in these studies, either with or without random group assignment. Randomized controlled trials are considered the "gold standard."
• Meta-analyses and systematic reviews – These studies aggregate and analyze data from multiple previous trials.

Each type has particular strengths and limitations. For example, observational studies show associations but cannot prove causation, while experimental studies test cause-and-effect but often have limited real-world applicability.

Evaluating Observational and Experimental Studies

When assessing observational research, readers should watch for potential biases and confounding factors that might skew results. Examples include:

 
• Healthy user bias – Health-conscious people may have many positive behaviors that set them apart from less health-conscious people, beyond just the factor being studied.
• Recall bias – People inaccurately remember past behaviors and exposures when asked to self-report them later.

Likewise, while experimental trials are more scientifically rigorous, issues can still arise around:

 
• Randomization – Was group assignment truly random?
• Blinding – Did subjects and researchers know which interventions each group received?
• Outcomes – Were outcomes pre-specified, measurable, and clinically meaningful?
• Power – Was the study adequately powered to detect differences between groups?
• Measurement – Could results be explained by bias or performance effects?
• Conflicts of interest – Did funding sources or author relationships influence the trial?
• Generalizability – How well does the study population reflect real-world circumstances?

Phases of Clinical Trials

When testing new drugs or devices, researchers conduct studies in four phases:

 
1. Phase 1 trials – Small studies of up to 100 people evaluating safety and dosage.
2. Phase 2 trials – Studies of up to several hundred people assessing efficacy and continued safety.
3. Phase 3 trials – Large, controlled studies of up to thousands of subjects measuring real world efficacy.
4. Phase 4 trials – Post-approval studies monitoring effects in wider population groups.

Authorities can halt trials at any phase if significant safety issues arise or if it becomes mathematically futile to continue because an intervention is unlikely to ever show benefit.

Key Statistical Concepts in Medical Literature

Understanding key statistical terminology and measures enables readers to accurately interpret study findings:

 
• P-values – The probability of seeing a result by chance if the null hypothesis of no effect were actually true. By convention, p≤0.05 is the threshold for statistical significance.
• Confidence intervals – The range of values within which researchers can be fairly confident the true population average lies. Narrower intervals indicate more precision.
• Hazard ratios – Compare the risk of an event occurring between two groups over time. A hazard ratio above 1 indicates increased risk in the intervention group while below 1 indicates decreased risk.
• Number needed to treat (NNT) – The number of people who would need to receive an intervention for one person to see a desired benefit. Lower NNTs indicate more powerful interventions.

Medical Journal Publication Process

After completing a study, authors submit manuscripts describing their research to peer-reviewed journals. Editors first assess if the topic fits within the journal’s scope and interests. If so, they invite expert reviewers to critique the methods, analyses, and conclusions. Reviewers determine if papers should be accepted, rejected, or revised and resubmitted. Studies with negative or undesirable results face barriers to publication, however, known as publication bias. Initiatives to combat this include requiring pre-registration of trial protocols ahead of time so journals expect all results.

Principles for Reading and Analyzing Papers

When reviewing medical literature, first read the abstract to gauge interest and consider background knowledge on the topic. For unfamiliar areas, read the introduction before diving into details on methods and results. Pay particular attention to figures, which should stand alone in explaining key data. Finally, read the discussion to compare the authors’ interpretation with your own critical analysis. Referencing the criteria above allows readers to systematically assess the strengths and weaknesses of research studies.

By understanding the scientific process, types of studies, and key statistical concepts, combined with rigorously evaluating studies for limitations, readers can separate reliable medical insights from speculation and noise.