Week 1: Systematic reviews and observational studies

SKI3010 · Wed 27 Jan

Lecture

  • Course overview
  • Systematic literature reviews
  • Observational study designs
  • Risk of bias and the Newcastle-Ottawa Scale (NOS)
  • Data extraction: study characteristics
  • Example primary paper

Slides and materials

Lecture slides and other materials appear here before the lecture.

Tutorial

  • Introduction: systematic review and research topic
  • Search and find relevant papers (different databases)
  • Choose your partner

Homework

  • Read the 6 primary studies and make a PRISMA flow chart
  • Read: Chapter 11, Textbook of Epidemiology
  • Install: Zotero

Concepts and practice

These concepts are covered in Quiz 1 (next week’s tutorial). Textbook: Chapters 4 and 11.

Concept list

Concept In one sentence
Narrative review A summary of the literature without a predefined, reproducible method for finding and selecting studies.
Systematic review A review that answers a focused question with a predefined, reproducible method to find, select, appraise and synthesise all relevant studies.
Meta-analysis The statistical pooling of the results of the studies included in a systematic review into one summary estimate.
PECO / PICO The elements of a focused review question: Population, Exposure (or Intervention), Comparison, Outcome.
Eligibility criteria The predefined inclusion and exclusion criteria a study must meet to be included in the review.
Search strategy The combination of databases, search terms and Boolean operators used to find all potentially relevant studies.
Screening Selecting studies in two steps: first on title and abstract, then on full text.
PRISMA 2020 The reporting guideline for systematic reviews, including the flow diagram of identified, screened and included studies.
Cohort study An observational study that follows exposed and unexposed people over time and compares how often the outcome occurs.
Case-control study An observational study that compares the past exposure of people with the outcome (cases) to people without it (controls).
Cross-sectional study An observational study that measures exposure and outcome at the same moment.
Observational versus experimental In an observational study the researcher does not assign the exposure; in an experiment (such as a randomised controlled trial) the researcher does.
Hierarchy of evidence The ranking of study designs by how well they support causal conclusions.

Practice questions

1. A researcher selects 200 children with autism and 400 children without autism and asks their parents which vaccines the child received. Which study design is this, and why?

A case-control study: participants are selected on the outcome (autism or not), and past exposure (vaccination) is then compared between cases and controls.

2. Formulate the review question of this module in PECO format.

P: children; E: thimerosal-containing vaccines; C: thimerosal-free vaccines (or lower exposure); O: autism spectrum disorder.

3. Why is a meta-analysis not always possible, even after a good systematic review?

Because the included studies may be too different (in population, exposure, outcome or design) or may not report the data needed to pool their results. The review then stays a systematic review with a narrative synthesis.

Practice quiz

The real quiz has the same format: 15 minutes, on paper, no devices (a simple calculator is allowed). Try it without your notes first.

  1. Name the four elements of a PECO question. (1 point)
  2. A study follows 10,000 children from birth and records which vaccines they receive and whether they are later diagnosed with autism. Which design is this? (1 point)
  3. Give two differences between a narrative review and a systematic review. (2 points)
  4. In a PRISMA flow diagram, 412 records are identified, 37 duplicates are removed, 340 records are excluded on title and abstract, and 26 full texts are excluded. How many studies are included? (2 points)
  5. Why do systematic reviews on vaccines and autism rely mostly on observational studies and not on randomised trials? (2 points)
  1. Population, Exposure, Comparison, Outcome.
  2. A (prospective) cohort study.
  3. Any two of: a systematic review has a predefined and reproducible search, explicit eligibility criteria, a risk-of-bias assessment of each study, and a transparent synthesis; a narrative review does not.
  4. 412 − 37 = 375 screened; 375 − 340 = 35 full texts; 35 − 26 = 9 studies included.
  5. Withholding or randomly assigning vaccines to children is not ethical or feasible, and autism is rare, so very large populations and long follow-up are needed. Observational cohort and case-control studies are then the best available evidence.