goodthingswithdata.comExemplar presentations · 2 of 7

Statistical decisions: the elements of testing hypotheses

Chad Heilig — EIS Summer Course, 2002-07-11 — 142 slides

This rendering was constructed from an original PowerPoint presentation. Corrections of original text are indicated as “[corrected: X]”, redactions as “[redacted: X]”, and names withheld as “[generalized: X]”. Handouts and other auxiliary sources mentioned in the presentations are not available. The content of each presentation was believed to be correct and valid at the time of the presentation. This content, including references and links within the slides, has not been checked or updated when preparing this site. This rendering was facilitated by Anthropic Claude agentic tools for automated text extraction through computer vision and for assistance with compiling the completed sets.

Setting: Statistical decisions: the elements of testing hypotheses, 2002-07-11. CDC’s Epidemic Intelligence Service (EIS) summer course, 70+ incoming postdoctoral fellows, mostly physicians and epidemiologists.

They understood less after my lecture than before it. And not in some appreciate-what-you-have-yet-to-learn kind of way. Among respondents, 40% said that they had basic, little, or no knowledge or skill in the subject matter before my seminar. After the seminar, that number climbed to 57% – a net loss of 17%.

I had led a 3-hour seminar titled “Statistical decisions: the elements of testing hypotheses” for CDC’s EIS summer course, populated by several dozen incoming postdoctoral fellows – physicians and epidemiologists and other learners. I’d poured my heart into a session motivated by intuition and followed by practical rigor.

Join me in my supervillain origin story, through the vicissitudes of selected evaluations:

In fairness, I had quoted Alan Greenspan: “If I have made myself clear, you must have misunderstood me.” I threw this in too, with apologies: “All we are saying is give chance some Ps”. Slide 53
This polarizing presentation is an important part of my intellectual history. Since then, I have turned my teaching and mentoring ways to the cause of good rather than evil. I have won plaudits from learners and the EIS program itself. And I have envisioned a whole progressive culture for doing good things with data.

Works referred to

Full references: this page · whole site, by domain (opens in a new tab) · whole site, by type of work (opens in a new tab)

Slide 1Statistical Decisions: the elements of testing hypotheses

Blue title slide with light rays; HHS seal lower left, CDC logo lower right, tagline band at bottom; a redacted region below the affiliation.

Chad Heilig
Division of Reproductive Health
Information Technology, Statistics, and Surveillance Branch
[redacted: email address]

Description

Text in the picture

DEPARTMENT OF HEALTH & HUMAN SERVICES • USA
CDC
SAFER • HEALTHIER • PEOPLE™

Abbreviations: CDC = Centers for Disease Control and Prevention; HHS = Department of Health and Human Services; USA = United States of America.

Slide 2Thesis

Slide 2: Thesis; its text follows
  • When properly done, statistical hypothesis tests serve one function
  • To distinguish chance influences from non-chance influences
  • With long-term control on the rate of incorrect inferences
  • Nothing less
  • Equally important, nothing more

Slide 3Objectives

Slide 3: Objectives; its text follows
  • Define the components of an appropriate statistical test
  • Define the P-value and note its limitations
  • Identify and interpret several common types of hypothesis tests
  • Address practical and ethical issues in hypothesis testing
Presenter notes

Emphasize intuition over derivation, practicality over theory, interpretation over calculation

Slide 4Outline

Slide 4: Outline; its text follows
  • Part 1
  • Elements of a hypothesis test
    • BMI and HIV in women
  • Part 2
  • Common tests in epidemiology
  • Part 3
  • Hypothesis testing in context

Abbreviations: BMI = body mass index (kg/m²); HIV = human immunodeficiency virus.

Slide 5Part 1: Elements of a hypothesis test

Slide 5: Part 1: Elements of a hypothesis test; its text follows

Example: body mass index (BMI: kg/m2) and
human immunodeficiency virus (HIV)
in women

Presenter notes

Define BMI: mass in kilograms divided by height in meters, squared.

Abbreviations: BMI = body mass index (kg/m²); HIV = human immunodeficiency virus.

Slide 6[quotations]

Slide 6: [quotations]; its text follows
  • Science may be described as the art of systematic over-simplification.
  • — Karl Popper

Slide 7Scientific context: substantive hypothesis

Slide 7: Scientific context: substantive hypothesis; its text follows
  • Substantive hypothesis
    • HIV, along with directly related conditions, induces higher metabolism and lower absorption in women.

Abbreviations: HIV = human immunodeficiency virus.

Slide 8Scientific context: research hypothesis

Slide 8: Scientific context: research hypothesis; its text follows
  • Substantive hypothesis
  • Research hypothesis
    • If HIV induces higher metabolism and lower absorption,
  • then women with HIV will experience greater abnormalities in metabolism and absorption than HIV-uninfected women.

Abbreviations: HIV = human immunodeficiency virus.

Slide 9Scientific context: Experimental hypothesis

Slide 9: Scientific context: Experimental hypothesis; its text follows
  • Substantive hypothesis
  • Research hypothesis
  • Experimental hypothesis
    • If HIV+ women experience greater abnormalities in metabolism and absorption than HIV– women,
  • then HIV+ women will have lower average BMI than HIV– women.

Abbreviations: BMI = body mass index (kg/m²); HIV = human immunodeficiency virus; HIV+ = HIV-positive.

Slide 10Scientific context: Statistical hypothesis

Slide 10: Scientific context: Statistical hypothesis; its text follows
  • Substantive hypothesis
  • Research hypothesis
  • Experimental hypothesis
  • Statistical hypothesis
    • If HIV+ women have lower average BMI than HIV– women,
    • then HIV+ women will have a mean BMI < 26.2.
    • Otherwise, HIV+ women will have a mean BMI ≥ 26.2.
Presenter notes

Suppose the population average BMI for US women is 26.2.

Abbreviations: BMI = body mass index (kg/m²); HIV = human immunodeficiency virus; HIV+ = HIV-positive; US = United States.

Slide 11Statistical hypothesis (1)

Slide 11: Statistical hypothesis (1); its text follows
  • Statistical hypothesis
    • HIV+ women will have an average BMI < 26.2.
    • Consequent of a conditional statement
  • Statement about unknown population mean of HIV+ women
    • Mean
    • Population
    • Unknown (unknowable?)

Abbreviations: BMI = body mass index (kg/m²); HIV+ = HIV-positive.

Slide 12Statistical hypothesis (2)

Slide 12: Statistical hypothesis (2); its text follows
  • Statement about unknown population mean of HIV+ women
    • Mean
    • Population
    • Unknown
  • Draw random sample
  • Observe BMI in sample
  • Relate back to population
Presenter notes

This sample is not really random

Abbreviations: BMI = body mass index (kg/m²); HIV+ = HIV-positive.

Slide 13Sample data

Slide 13: Sample data; its text follows
  • The HIV Epidemiology Research (HER) Study
  • 871 HIV+, 439 HIV– women
  • “High risk” by well-defined criteria
  • 4 US communities
  • Every 6 months, up to 15 visits
  • Visits 1, 3: 617 HIV+, 318 HIV–
Presenter notes

A sample of US women with HIV

Abbreviations: HER/HERS = HIV Epidemiology Research (HER) Study; HIV = human immunodeficiency virus; HIV+ = HIV-positive; US = United States.

Slide 14Figure: distribution of BMI in HIV+ HERS women at enrollment (1)

Rug plot of BMI values as red tick marks along a horizontal axis, with the area above it empty, on a white background.
Description
Panels
1
Elements
A rug of thin vertical red tick marks in a single row near the bottom depicts individual BMI values; overlapping ticks appear darker
Horizontal
BMI, 15 to 50; ticks extend from about 16 to about 49
Vertical
No scale; the ticks occupy a narrow strip just above the axis, and the space above them, is empty
Visible pattern
The ticks are densest and darkest from about 18 to about 28, and they thin out from about 28 to about 37 and then become sparse.

Text in the picture

Distribution of BMI in HIV+ HERS women at enrollment

Abbreviations: BMI = body mass index (kg/m²); HER/HERS = HIV Epidemiology Research (HER) Study; HIV+ = HIV-positive.

Slide 15Figure: distribution of BMI in HIV+ HERS women at enrollment (2)

The same rug plot of BMI values, now with a small table of summary statistics at upper right.
Description
Panels
1
Elements
A rug of thin vertical red tick marks in a single row near the bottom depicts individual BMI values; a small table of summary statistics in red and black sits at upper right
Horizontal
BMI, 15 to 50; ticks extend from about 16 to about 49
Vertical
No scale; the ticks occupy a narrow strip just above the axis; the space above them is empty apart from the table
Visible pattern
The ticks are densest and darkest from about 18 to about 28, and they thin out from about 28 to about 37 and then become sparse.

Text in the picture

Distribution of BMI in HIV+ HERS women at enrollment
N mean SD skew
HIV+ 617 25.7 6.29 1.10

Abbreviations: BMI = body mass index (kg/m²); HER/HERS = HIV Epidemiology Research (HER) Study; HIV+ = HIV-positive; SD = standard deviation.

Slide 16Figure: distribution of BMI in HIV+ HERS women at enrollment (3)

Rug plot of BMI values with a smoothed red density curve above it and a summary table at upper right.
Description
Panels
1
Elements
1 smoothed red curve depicts the shape of the BMI distribution, above a rug of thin vertical red tick marks depicting individual BMI values; a small table of summary statistics at upper right
Horizontal
BMI, 15 to 50; the curve spans the full extent, ticks from about 16 to about 49
Vertical
No scale or axis; the curve fills most of the plot height above the rug, peaking near the top
Visible pattern
The density curve peaks near BMI 22, then falls more gradually to about 40 and flattens into a long, low tail.

Text in the picture

Distribution of BMI in HIV+ HERS women at enrollment
N mean SD skew
HIV+ 617 25.7 6.29 1.10

Abbreviations: BMI = body mass index (kg/m²); HER/HERS = HIV Epidemiology Research (HER) Study; HIV+ = HIV-positive; SD = standard deviation.

Slide 17A hypothetical sample

Slide 17: A hypothetical sample; its text follows
  • Suppose we take a sample from a hypothetical population with mean BMI 26.2 and SD 6.29.
  • This is simulated in the computer
    • Draw 617 observations from this population
    • Observed sample mean is 26.361
  • Repeat

Abbreviations: BMI = body mass index (kg/m²); SD = standard deviation.

Slide 18Hypothetical population

A pointing-hand symbol, pointing right, follows the word Repeat at lower right.
  • Take samples of 617 women
  • Compute mean for each sample
SampleMean
126.361
226.158
325.876
426.477
526.066

Repeat →

Presenter notes

Because the HIV+ population mean is unknown

Abbreviations: HIV+ = HIV-positive.

Slide 19Simulated sample means (1)

Rug plot: a row of thin vertical ticks above an axis of BMI sample mean, with a small summary table at upper right.
Description
Panels
1
Elements
1 rug of thin pink vertical ticks depicts the simulated sample means; no curve
Horizontal
BMI sample mean, 25.4 to 27.0 in steps of 0.2; ticks run from about 25.55 to about 27.06, just past the end of the axis
Vertical
No vertical axis or scale; the ticks occupy a narrow band just above the horizontal axis, and the space above them is empty apart from the table
Visible pattern
The ticks crowd most densely between about 25.8 and 26.5 and thin out toward both ends. The right end trails further than the left, with widely spaced ticks from about 26.6 to just past 27.0, while the left end stops near 25.55 after a few scattered ticks.

Text in the picture

Simulated sample means from hypothetical population
150 samples, each with n=617
x̄
N mean SD skew
Population 617 26.2 6.286 1.10
Sample means
Empirical 150 26.2 0.260 0.20

Abbreviations: BMI = body mass index (kg/m²); SD = standard deviation.

Slide 20Simulated sample means (2)

A single density curve over a rug of ticks for simulated BMI sample means, with a small summary table at upper right.
Description
Panels
1
Elements
1 solid dark red density curve above a rug of thin pink vertical ticks; both depict simulated sample means
Horizontal
BMI sample mean, 25.4 to 27.0 in steps of 0.2; curve and ticks run from about 25.55 to about 27.06, just past the end of the axis
Vertical
No vertical axis or scale; the curve's peak reaches near the top of the plotting area
Visible pattern
The curve is single-humped with a broad, nearly flat top spanning about 26.1 to 26.3. It rises more steeply on the left, where it ends near 25.55, and falls more gradually on the right, trailing to about 27.06. The ticks beneath are densest between about 25.8 and 26.5 and sparse at both ends.

Text in the picture

Simulated sample means from hypothetical population
1000 samples, each with n=617
x̄
N mean SD skew
Population 617 26.2 6.286 1.10
Sample means
Empirical 1000 26.2 0.252 0.13

Abbreviations: BMI = body mass index (kg/m²); SD = standard deviation.

Slide 21Simulated sample means (3)

Two overlaid density curves, solid and dashed, over a rug of ticks for simulated BMI sample means, with a summary table and an orange formula.
Description
Panels
1
Elements
2 dark red density curves (1 solid, 1 dashed) above a rug of thin pink vertical ticks; both density curves and ticks depict sample means; a faint gray dashed vertical reference line
Horizontal
BMI sample mean, 25.4 to 27.0 in steps of 0.2; reference line at 26.2; the solid curve runs from about 25.55 to about 27.06, the dashed curve from about 25.3 to past 27.0, beyond both ends of the axis
Vertical
No vertical axis or scale; the dashed curve's peak reaches the top of the plotting area
Visible pattern
The dashed curve is a smooth symmetric bell centered on the reference line at 26.2 and peaks above the solid curve. The solid curve follows it closely over most of the range: lower and flatter at the top, slightly outside it on the left flank, and slightly above it in the right tail from about 26.7 onward. The dashed curve extends past both ends of the solid curve.

Text in the picture

Simulated sample means from hypothetical population
1000 samples, each with n=617
x̄
N mean SD skew
Population 617 26.2 6.286 1.10
Sample means
Empirical 1000 26.2 0.252 0.13
Theoretical Inf 26.2 0.253 0.00
6.286 / √617 = 0.253

Presenter notes

Be clear on SD and SE

Abbreviations: BMI = body mass index (kg/m²); SD = standard deviation; SE = standard error.

Slide 22Simulated sample means (4)

Two overlaid density curves, solid and dashed, over a rug of ticks for sample means shifted to center on zero, with a summary table.
Description
Panels
1
Elements
2 dark red density curves (1 solid, 1 dashed) above a rug of thin pink vertical ticks depict shifted sample means; a faint gray dashed vertical reference line
Horizontal
Labeled BMI sample mean, -0.8 to 0.8 in steps of 0.2; reference line at 0.0; ticks run from about -0.65 to about 0.86; the dashed curve extends past both ends of the axis
Vertical
No vertical axis or scale; the dashed curve's peak reaches the top of the plotting area
Visible pattern
The same 2 curves as before, now centered on the reference line at 0.0: a smooth symmetric dashed bell and a solid curve that is lower and flatter on top, slightly outside the bell on the left flank and slightly above it in the right tail. The ticks are densest between about -0.4 and 0.3, with the right end trailing further than the left.

Text in the picture

Simulated sample means from hypothetical population
1000 samples, each with n=617, shifted by 26.2
x̄ − 26.2
N mean SD skew
Population 617 26.2 6.286 1.10
Sample means
Empirical 1000 0.0 0.252 0.13
Theoretical Inf 0.0 0.253 0.00

Abbreviations: BMI = body mass index (kg/m²); SD = standard deviation.

Slide 23Simulated sample means (5)

Two overlaid density curves, solid and dashed, over a rug of ticks for standardized sample means centered on zero, with a summary table and formula.
Description
Panels
1
Elements
2 dark red density curves (1 solid, 1 dashed) above a rug of thin pink vertical ticks depict sample means; a faint gray dashed vertical reference line
Horizontal
Labeled BMI sample mean, -3.5 to 3.5 in steps of 0.5; reference line at 0.0; ticks run from about -2.55 to about 3.4; the dashed curve spans the full axis
Vertical
No vertical axis or scale; the dashed curve's peak reaches the top of the plotting area
Visible pattern
The 2 curves keep their shapes, centered on 0.0, with the dashed bell symmetric and the solid curve lower and flatter on top and slightly heavier in the right tail. The dashed curve approaches zero height near both ends of the axis. The ticks are densest between about -1.5 and 1.2, with a longer trail to the right.

Text in the picture

Simulated sample means from hypothetical population
1000 samples, each with n=617, shifted by 26.2, divided by 0.253
(x̄ − 26.2) / 0.2531
N mean SD skew
Population 617 26.2 6.286 1.10
Sample means
Empirical 1000 0.0 0.997 0.13
Theoretical Inf 0.0 1.000 0.00

Abbreviations: BMI = body mass index (kg/m²); SD = standard deviation.

Slide 24Simulated sample means (6)

Standardized solid and dashed density curves over a rug of ticks, with the left tail below a cutoff shaded gray and labeled 5%.
Description
Panels
1
Elements
2 dark red density curves (1 solid, 1 dashed) above a rug of thin pink vertical ticks; a gray-filled area under the dashed density curve; 2 faint gray dashed vertical reference lines; an orange label with an orange arrow pointing into the gray area
Horizontal
Labeled BMI sample mean, -3.5 to 3.5 in steps of 0.5; reference lines at 0.0 and at about -1.645; the gray area spans from the left end of the axis to the line at about -1.645
Vertical
No vertical axis or scale; the dashed curve's peak reaches the top of the plotting area
Visible pattern
The curves and ticks are as on the previous slide. The gray area fills the left tail under the dashed curve, rising from near zero height at -3.5 to about a quarter of the peak height at the cutoff near -1.645.

Text in the picture

Simulated sample means from hypothetical population
1000 samples, each with n=617, shifted by 26.2, divided by 0.253
(x̄ − 26.2) / 0.2531
−1.645
5%
N mean SD skew
Population 617 26.2 6.286 1.10
Sample means
Empirical 1000 0.0 0.997 0.13
Theoretical Inf 0.0 1.000 0.00

Abbreviations: BMI = body mass index (kg/m²); SD = standard deviation.

Slide 25Simulated sample means (7)

Standardized solid and dashed density curves with the left tail shaded gray, 2 cutoff lines, and an orange calculation at left.
Description
Panels
1
Elements
2 dark red density curves (1 solid, 1 dashed) above a rug of thin pink vertical ticks; a gray-filled area under the dashed density curve; 3 dashed vertical reference lines, 2 faint gray and one darker inside the gray area
Horizontal
Labeled BMI sample mean, -3.5 to 3.5 in steps of 0.5; reference lines at 0.0, about -1.645 and about -1.93; the gray area spans from the left end of the axis to about -1.645
Vertical
No vertical axis or scale; the dashed curve's peak reaches the top of the plotting area
Visible pattern
The gray left tail ends at the cutoff near -1.645, as before. A second line, at about -1.93, lies a short distance further left and passes through the gray area, where the curve is a little lower. The label for -1.930 is black and the label for -1.645 is gray.

Text in the picture

Simulated sample means from hypothetical population
1000 samples, each with n=617, shifted by 26.2, divided by 0.253
(x̄ − 26.2) / 0.2531
(25.711 − 26.2) / 0.2531 = −1.930
−1.930
−1.645
N mean SD skew
Population 617 26.2 6.286 1.10
Sample means
Empirical 1000 0.0 0.997 0.13
Theoretical Inf 0.0 1.000 0.00

Abbreviations: BMI = body mass index (kg/m²); SD = standard deviation.

Slide 26Simulated sample means (8)

Standardized density curves with the left tail split into pink and gray regions at 2 cutoffs, the pink region labeled 2.7%.
Description
Panels
1
Elements
2 dark red density curves (1 solid, 1 dashed) above a rug of thin pink vertical ticks; under the dashed density curve a pink-filled area and a gray-filled area; 3 faint dashed vertical reference lines; an orange label with an orange arrow pointing into the pink area
Horizontal
Labeled BMI sample mean, -3.5 to 3.5 in steps of 0.5; reference lines at 0.0, about -1.645 and about -1.93; pink area from the left end of the axis to about -1.93, gray area from about -1.93 to about -1.645
Vertical
No vertical axis or scale; the dashed curve's peak reaches the top of the plotting area
Visible pattern
The left tail under the dashed curve is now divided at the line near -1.93: the larger, lower part to its left is pink, and a narrow gray strip remains between -1.93 and -1.645. The rest of the plot is unchanged from the previous slide.

Text in the picture

Simulated sample means from hypothetical population
1000 samples, each with n=617, shifted by 26.2, divided by 0.253
(x̄ − 26.2) / 0.2531
(25.711 − 26.2) / 0.2531 = −1.930
−1.930
−1.645
2.7%
N mean SD skew
Population 617 26.2 6.286 1.10
Sample means
Empirical 1000 0.0 0.997 0.13
Theoretical Inf 0.0 1.000 0.00

Abbreviations: BMI = body mass index (kg/m²); SD = standard deviation.

Slide 27Distribution of BMI

A single density curve over a rug of ticks showing a right-skewed distribution of BMI, with a one-row summary table at upper right.
Description
Panels
1
Elements
1 solid red density curve above a rug of thin red vertical ticks; both depict BMI values
Horizontal
BMI, 15 to 50 in steps of 5; the curve spans the full axis; ticks run from about 16 to about 49.5
Vertical
No vertical axis or scale; the curve's peak reaches near the top of the plotting area
Visible pattern
The curve rises steeply from the left end of the axis to a single peak near 21.5, then falls more slowly, with slight shoulders near 26 and 31. Beyond about 38 it flattens into a long, low right tail that declines gently to 50. The ticks are dense from about 16 to 33, thinner to about 37, and widely scattered from about 38 to 49.5.

Text in the picture

Distribution of BMI in HIV+ HERS women at enrollment
N mean SD skew
HIV+ 617 25.7 6.29 1.10

Abbreviations: BMI = body mass index (kg/m²); HER/HERS = HIV Epidemiology Research (HER) Study; HIV+ = HIV-positive; SD = standard deviation.

Slide 28Standardized sampling distribution

A smooth symmetric bell curve with the left tail split into pink and gray regions at 2 labeled cutoffs.
Description
Panels
1
Elements
1 solid dark red bell curve; under it a pink-filled area and a gray-filled area; 3 faint dashed vertical reference lines; no rug and no table
Horizontal
Labeled BMI sample mean, -3.5 to 3.5 in steps of 0.5; reference lines at 0.0, about -1.645 and about -1.93; pink area from the left end of the axis to about -1.93, gray area from about -1.93 to about -1.645
Vertical
No vertical axis or scale; the curve's peak reaches the top of the plotting area
Visible pattern
The curve is a single symmetric bell centered on the line at 0.0, approaching zero height at both ends of the axis. The left tail beyond about -1.93 is pink, and a narrow gray strip lies between -1.93 and -1.645, where the curve stands at about a quarter of its peak height.

Text in the picture

Standardized sampling distribution
−1.930
−1.645

Abbreviations: BMI = body mass index (kg/m²).

Slide 29Recapitulation of example (1)

Slide 29: Recapitulation of example (1); its text follows
  • Chain of if-then hypotheses
    • If HIV induces higher metabolism and lower absorption, then … HIV+ women will have a mean BMI < 26.2.
  • Under the hypothesis that true mean=26.2 (assumed SD=6.29)
    • Sample mean is 1.93 SDs below hypothetical mean.
    • A value lower than 1.645 SDs below the hypothetical mean should occur in only about 5% of samples.

Abbreviations: BMI = body mass index (kg/m²); HIV = human immunodeficiency virus; HIV+ = HIV-positive; SD = standard deviation.

Slide 30Recapitulation of example (2)

Slide 30: Recapitulation of example (2); its text follows
    • If HIV induces higher metabolism and lower absorption, then … HIV+ women will have a mean BMI < 26.2.
    • A value lower than 1.645 SDs below the hypothetical mean should occur in fewer than 5% of samples.
  • Either a rare event has occurred by chance,
  • or the low value is due to
  • non-chance factors
  • Conclude: non-chance factors

Abbreviations: BMI = body mass index (kg/m²); HIV = human immunodeficiency virus; HIV+ = HIV-positive; SD = standard deviation.

Slide 31Recapitulation of example (3)

Slide 31: Recapitulation of example (3); its text follows
    • If HIV induces higher metabolism and lower absorption, then … HIV+ women will have a mean BMI < 26.2.
    • A value lower than 1.645 SDs below the hypothetical mean should occur in fewer than 5% of samples.
    • Conclude: non-chance factors
  • Complete the chain of reasoning
    • If HIV+ mean BMI < 26.2, then …
  • Can’t logically complete the chain of reasoning

Abbreviations: BMI = body mass index (kg/m²); HIV = human immunodeficiency virus; HIV+ = HIV-positive; SD = standard deviation.

Slide 32Continuation of example

Slide 32: Continuation of example; its text follows
  • Suppose the hypothetical population mean were 26.0
  • How would our procedure turn out then?

Slide 33Simulated sample means (9)

A dashed bell curve centered at 26.0 with a vertical line marking a labeled sample mean to its left, and a summary table.
Description
Panels
1
Elements
1 dashed dark red bell curve; 2 faint gray dashed vertical reference lines, one with a stacked text label; no rug
Horizontal
BMI sample mean, 25.2 to 26.8 in steps of 0.2; reference lines at 26.0 and at about 25.71; the curve extends slightly past both ends of the axis
Vertical
No vertical axis or scale; the curve's peak reaches the top of the plotting area
Visible pattern
The curve is a single symmetric bell centered on the line at 26.0, approaching zero height near both ends of the axis. The labeled line at about 25.71 crosses the left flank at roughly half the peak height.

Text in the picture

Simulated sample means from hypothetical population, mean=26.0
Each with n=617
HERS
sample
mean
25.71
N mean SD
Population 617 26.0 6.286
Sample mean Inf 26.0 0.253

Abbreviations: BMI = body mass index (kg/m²); HER/HERS = HIV Epidemiology Research (HER) Study; SD = standard deviation.

Slide 34Simulated sample means (10)

Standardized dashed bell curve with the left tail shaded gray and a pink band beside it, labeled 13%, and an orange calculation.
Description
Panels
1
Elements
1 dashed dark red bell curve; under it a gray-filled area and a pink-filled area; 3 faint dashed vertical reference lines; an orange label with 2 orange arrows, one pointing into the gray area and one to the top of the boundary between the gray and pink areas
Horizontal
Labeled BMI sample mean, -3.5 to 3.5 in steps of 0.5; reference lines at 0.0, about -1.645 and about -1.146; gray area from the left end of the axis to about -1.645, pink area from about -1.645 to about -1.146
Vertical
No vertical axis or scale; the curve's peak reaches the top of the plotting area
Visible pattern
The curve is a single symmetric bell centered on the line at 0.0. The gray left tail ends at about -1.645, and a pink band continues to the right of it up to the line at about -1.146, where the curve stands at roughly half its peak height. Here the line at -1.146 lies to the right of the line at -1.645, and its label is black while the -1.645 label is gray.

Text in the picture

Simulated sample means from hypothetical population, mean=26.0
Each with n=617, shifted by 26.0, divided by 0.253
(x̄ − 26.0) / 0.2531
(25.711 − 26.0) / 0.2531 = −1.146
−1.645
−1.146
13%
N mean SD
Population 617 26.0 6.286
Sample mean Inf 0.0 1.000

Abbreviations: BMI = body mass index (kg/m²); SD = standard deviation.

Slide 35Completion of example (1)

Slide 35: Completion of example (1); its text follows
  • If population mean is 26.0, sample mean of 25.7 does not seem extreme: about 1.15 SDs below the hypothetical mean
  • Sample mean is “within the noise”; reasonably consistent with 26.0
  • What does a population
  • BMI ≥ 26.0 imply?

Abbreviations: BMI = body mass index (kg/m²); SD = standard deviation.

Slide 36Completion of example (2)

Slide 36: Completion of example (2); its text follows
  • If population BMI for HIV+ women HIV is ≥ 26.0
    • HIV+ women do not have lower average BMI than HIV– women.
    • HIV+ women do not experience greater abnormalities in metabolism and absorption than HIV– women.
    • HIV, along with directly related conditions, does not induce higher metabolism and lower absorption in women.

Abbreviations: BMI = body mass index (kg/m²); HIV = human immunodeficiency virus; HIV+ = HIV-positive.

Slide 37Features of procedure

Slide 37: Features of procedure; its text follows
  • Establish scientific context
  • Formulate statistical hypothesis
  • Design sampling procedure
  • Set a tolerable error rate
  • Identify rejection values
  • Obtain and use sample
  • Test hypotheses

Slide 38Establish scientific context

Slide 38: Establish scientific context; its text follows
  • Substantive hypothesis (SubH)
    • The to-be-explained phenomenon
  • Research hypothesis (ResH)
    • Theoretical implication of SubH
  • Experimental hypothesis (ExpH)
    • Implication of ResH subject to (manipulation and) measurement
  • Statistical hypothesis (StaH)
    • Quantitative implication of ExpH

Abbreviations: ResH = research hypothesis; StaH = statistical hypothesis; SubH = substantive hypothesis.

Slide 39Formulate statistical hypotheses

Slide 39: Formulate statistical hypotheses; its text follows
  • The alternative hypothesis, H1
    • Statement about the population that you want to “prove”
  • The null hypothesis, H0
      • Hypothesis of zero difference
      • To-be-nullified hypothesis
      • Simpler of 2 hypotheses
      • Hypothesis with more serious consequences if rejected
      • Explanation to discredit

Slide 40Design sampling procedure

Slide 40: Design sampling procedure; its text follows
  • Various sampling methods
  • Analyzing data from a study designed by someone else
  • Without some randomness in sampling, no valid statistical test
  • Sample statistic, such as mean
    • Properties amenable to analysis
  • when H0 is true
Presenter notes

Simple: subtract and divide
Complex: intensive simulations

Slide 41Set a tolerable error rate (1)

Slide 41: Set a tolerable error rate (1)
State of affairs / (Conditional)Conclusion
Reject H0Do not reject H0
H0 trueType 1 errorNo error
H1 trueNo errorType 2 error
Presenter notes

Conditional probability

Slide 42Set a tolerable error rate (2)

Slide 42: Set a tolerable error rate (2); its text follows
  • Reject H0 when it is true
    • Type 1 error
    • α: Pr(reject H0 | H0 true)
      • In the long-run, repeated sampling under H0 would produce a Type 1 error in α100% samples
        • α=0.05 → 5% of samples
      • Also called “significance level”
    • Confidence level: (1–α)100%
  • Set α before test
Presenter notes

Conditional probability

Slide 43Identify rejection values

Slide 43: Identify rejection values; its text follows
  • Identify rejection region
    • All values of the test statistic for which H0 would be rejected
    • Depends on
      • Sampling properties of test statistic under H0
      • Selected confidence level
  • Often summarized by 1 or 2 cutpoints (example: x < –1.645)

Slide 44Obtain and use sample

Slide 44: Obtain and use sample; its text follows
  • Obtain sample
  • Compute sample statistic
  • Will almost certainly differ numerically from population parameter under H0

Slide 45Test hypotheses

Slide 45: Test hypotheses; its text follows
  • Relate test statistic to rejection region
    • Inside region: reject H0
  • Observed a rare phenomenon; difference from hypothesized parameter is due to factors other than chance
    • Outside region: do not reject H0
  • Difference from hypothesized parameter is due to chance
  • Relate to antecedent hypotheses

Slide 46Two more test features

Slide 46: Two more test features; its text follows
  • P-values
  • Failing to reject H0 when it is false

Slide 47P-values

Slide 47: P-values; its text follows
  • Given that H0 is true, probability of observing a test statistic at least as extreme as the one observed
  • “Attained significance level”
    • Smallest α for which the test statistic would indicate rejecting H0
  • P is a property of the sample;
  • α is a property of the hypothetical population
Presenter notes

Conditional probability

Slide 48The other testing error

Slide 48: The other testing error; its text follows
  • Accept H0 when H1 is true
    • Type 2 error
    • β: Pr(do not reject H0 | H1 true)
      • In the long-run, repeated sampling under H1 would produce a Type 2 error in β100% samples
        • β=0.20 → 20% of samples
    • Power: (1–β)100%
    • Calculable for simple H1 (μ=0)
  • Set β before test
Presenter notes

Conditional probability

Slide 49Example of beta (1)

A bell-shaped curve centered at 100, its left tail below about 93 shaded dark gray and labeled 5% by an orange arrow.
Description
Panels
1
Elements
1 bell-shaped curve; a dashed vertical line at the curve's center and a second dashed vertical line at a cutpoint to its left; the area under the curve left of the cutpoint is shaded dark gray, with an orange arrow pointing into it from the label "5%"
Horizontal
74 to 114 in steps of 2, unlabeled scale; the curve runs from about 86 to 114; dashed vertical guides at 100 and at about 93
Vertical
no axis, scale or labels
Visible pattern
A single symmetric curve rises from near zero at about 86 to its peak at 100 and falls back to near zero by 114. The thin left tail, from about 86 to the cutpoint at about 93, is shaded.

Text in the picture

Example of beta
SD(pop'n) 24
mean-null 100
N 36
SD(mean) 4
alpha 0.05

Abbreviations: SD = standard deviation.

Slide 50Example of beta (2)

Two identical bell-shaped curves centered at 92 and 100; the left tail of the curve at 100, below about 93, is shaded dark gray and labeled 5%.
Description
Panels
1
Elements
2 bell-shaped curves of the same width and height, each with a dashed vertical line at its center; a third dashed vertical line at a cutpoint; the area under the right-hand curve left of the cutpoint is shaded dark gray, with an orange arrow pointing into it from the label "5%"
Horizontal
74 to 114 in steps of 2, unlabeled scale; the curves together run from about 78 to 114; dashed vertical guides at 92, at about 93 and at 100
Vertical
no axis, scale or labels; both peaks at the same height
Visible pattern
The left curve spans about 78 to 106 and peaks at 92; the right curve spans about 86 to 114 and peaks at 100. The 2 curves overlap broadly between about 86 and 106 and cross at about 96. The cutpoint lies just right of the left curve's peak, and the dark gray tail of the right curve, from about 86 to 93, sits inside the left curve's central region.

Text in the picture

Example of beta
SD(pop'n) 24
mean-null 100
mean-alt 92
N 36
SD(mean) 4
alpha 0.05

Abbreviations: SD = standard deviation.

Slide 51Example of beta (3)

Two bell-shaped curves centered at 92 and 100, with a dark gray 5% tail under the right curve and a pale yellow 36.1% area under the left curve.
Description
Panels
1
Elements
2 bell-shaped curves of the same width and height, each with a dashed vertical line at its center; a third dashed vertical line at a cutpoint; under the right-hand curve, the area left of the cutpoint is shaded dark gray, with an orange arrow from the label "5%"; under the left-hand curve, the area right of the cutpoint is shaded pale yellow, with an orange arrow from the label "36.1%"; a parameter table is placed at the upper left.
Horizontal
74 to 114, ticks every 2; curves span about 78 to 114; thin dashed verticals at 92 and 100, bolder dashed vertical at about 93
Vertical
No axis or scale; both curves rise from a common baseline to the same peak height, with the parameter table to the left of the left curve's peak
Visible pattern
The curves are placed as on the previous slide: the left one spans about 78 to 106 and peaks at 92, the right one spans about 86 to 114 and peaks at 100, and they cross at about 96. The cutpoint at about 93 divides 2 shaded regions that meet along it: the small dark gray tail of the right curve to its left, from about 86, and the much larger yellow region of the left curve to its right, extending to about 106. The yellow region covers the left curve's whole right side beyond the cutpoint, including the area where the 2 curves overlap.

Text in the picture

Example of beta
SD(pop'n) 24
mean-null 100
mean-alt 92
N 36
SD(mean) 4
alpha 0.05
beta 0.361

Abbreviations: SD = standard deviation.

Slide 52Controlling error rates

Slide 52: Controlling error rates; its text follows
  • α and β are inversely related
    • for fixed sample size
  • Set α first
  • Then minimize β, subject to α and available resources
    • Part 3: 4 ways to reduce β
  • Report P after completing test
    • Other researchers can apply their own choice of confidence level

Slide 53[quotations]

Slide 53: [quotations]; its text follows
  • If I have made myself clear, you must have misunderstood me.
  • — Alan Greenspan
  • All we are saying is give chance some Ps.
  • (with apologies to John Lennon and Paul McCartney)

Slide 54Part 2: Common tests in epidemiology

Slide 54: Part 2: Common tests in epidemiology; its text follows

Slide 55[quotations]

Slide 55: [quotations]; its text follows
  • It is the mark of an educated mind to be able to entertain a thought without accepting it.
  • — Aristotle
  • All models are wrong but some are useful.
  • — George E P Box

Slide 56Test statistics redux

Slide 56: Test statistics redux; its text follows
  • Amenable to analysis under H0
  • Typically involves standardization
    • Observed minus expected, divided by SD of sample statistic
  • Could be computationally intensive
  • Distribution often given in tables
  • Better still: in computer software
Presenter notes

Observed minus expected (under H0), rescaled to unit variance (under H0)

Abbreviations: SD = standard deviation.

Slide 57Hypotheses redux

Slide 57: Hypotheses redux; its text follows
  • H0: null (to-be-nullified) hypothesis
  • H1: alternative hypothesis;
  • what you want to “prove”
  • 3 alternatives to H0: μ = 0
H1
Lower-tailedμ < 0
Upper-tailedμ > 0
Two-tailedμ ≠ 0

Slide 58Rejection regions for directional alternatives (1)

A bell-shaped curve centered on 0 with its left tail shaded, marking a 5% rejection region below −1.645.
Description
Panels
1
Elements
1 bell-shaped curve (thin black line); the area under its left tail is shaded dark gray, and an orange arrow labeled 5% points to the shaded area; orange labels give the hypotheses and the cutoff
Horizontal
-3.5 to 3.5, ticks every 0.5; the shaded area ends at -1.645, labeled beneath its right edge
Vertical
no axis or scale; the curve occupies the upper half of the plot area, and the lower half, above the horizontal axis, is empty
Visible pattern
The curve is symmetric, peaking above 0 and falling to a nearly flat baseline by -3.5 and 3.5. The shaded region is a thin wedge in the left tail, from -3.5 to about -1.6, rising toward its right edge where it meets the curve. Only the left side carries shading; the right tail is unshaded.

Text in the picture

Rejection regions for directional alternatives
H_0: μ = 0
H_lo: μ < 0
5%
−1.645

Slide 59Rejection regions for directional alternatives (2)

Two identical bell-shaped curves stacked one above the other: the upper with its left tail shaded, the lower with its right tail shaded.
Description
Panels
1
Elements
2 identical bell-shaped curves (thin black lines) offset vertically; dark gray shading under the left tail of the upper curve and under the right tail of the lower curve; orange arrows labeled 5% point to each shaded area; orange labels give the hypotheses and cutoffs
Horizontal
-3.5 to 3.5, ticks every 0.5; the upper curve's shading ends at -1.645 and the lower curve's begins at 1.645, each labeled
Vertical
no axis or scale; the 2 curves fill roughly the upper 3 quarters of the plot area, leaving empty space above the horizontal axis
Visible pattern
Both curves are symmetric and centered on 0, with the lower curve a copy of the upper one shifted down. The shaded regions mirror each other: a wedge from -3.5 to about -1.6 on the upper curve, and a wedge from about 1.6 to 3.5 on the lower curve. The hypothesis labels sit beside the tail they belong to, one at the left and one at the right.

Text in the picture

Rejection regions for directional alternatives
H_0: μ = 0
H_lo: μ < 0
5%
−1.645
H_up: μ > 0
5%
1.645

Slide 60Rejection regions for directional alternatives (2)

Three identical bell-shaped curves stacked vertically, shaded in the left tail, the right tail, and both tails.
Description
Panels
1
Elements
3 identical bell-shaped curves (thin black lines) offset vertically; dark gray shading under the left tail of the top curve, the right tail of the middle curve, and both tails of the bottom curve; orange arrows with percentage labels point to each shaded area; orange labels give the hypotheses and cutoffs
Horizontal
-3.5 to 3.5, ticks every 0.5; cutoffs labeled at -1.645 (top curve), 1.645 (middle curve), and -1.960 and 1.960 (bottom curve)
Vertical
no axis or scale; the bottom curve's baseline sits just above the horizontal axis
Visible pattern
All 3 curves are symmetric and centered on 0, each a copy of the one above shifted down by the same amounty. The top and middle curves each carry one wedge of shading, 5% each, in the left and right tails respectively, reaching in to about ±1.6. The bottom curve carries 2 smaller wedges, 2.5% each, beyond about ±2.0, with its hypothesis label placed beneath the center of the curve.

Text in the picture

Rejection regions for directional alternatives
H_0: μ = 0
H_lo: μ < 0
5%
−1.645
H_up: μ > 0
5%
1.645
2.5%
−1.960
H_2t: μ ≠ 0
2.5%
1.960

Slide 61Choosing a test

Slide 61: Choosing a test; its text follows
  • Features of data
    • Large or small amount of data?
    • Means, proportions, tables, other?
      • Odds ratios and prevalence ratios obtained from 2 x 2 tables
    • Single or multiple populations?
    • Assumptions about distribution(s)
      • Normal, not too non-normal
      • Minimal assumptions
Presenter notes

Not too non-normal: not too skewed, tails not too heavy; compensated by larger samples
If too non-normal: transformations, “distribution-free” tests

Slide 62Test for means

Slide 62: Test for means; its text follows
  • Comparing a single sample to a fixed standard?
    • Variance known or unknown?
  • Comparing two independent samples?
    • Variances known or unknown?
  • Variances equal or unequal?
  • Comparing two related samples (paired observations)?

Slide 63Specifying critical values

Slide 63: Specifying critical values; its text follows
  • Critical value zp is value of standard normal distribution such that Pr(Z ≤ zp) = p
  • Examples:
z0.025=–1.960
z0.050=–1.645
z0.950=1.645
z0.975=1.960

Slide 643 gifts from Guinness

The Guinness logo: a gold harp above the word GUINNESS between 2 thin gold rules, on a black background.
  • 1799: Guinness Porter Stout
  • 1955: The Guinness Book of Records
  • 1908: Student’s t test
Description

Text in the picture

17 59
GUINNESS

Slide 65Student’s t distribution

Slide 65: Student’s t distribution; its text follows
  • Distribution of sample mean when
    • Data come from normal distribution
    • Population variance unknown
  • Heavier tails, more variability
    • Dividing by an estimated standard deviation introduces more variability
  • Indexed by degrees of freedom (df)
Presenter notes

Degrees of freedom: # obs minus # independently estimated parameters
Invented by W S Gossett in 1908 at Guinness brewery, published under pseudonym at behest of brewery

Slide 66Student's t distribution on 5 degrees of freedom

A Student's t density on 5 degrees of freedom with a faint standard normal curve behind it; arrows mark 1.645 and 2.015.
Description
Panels
1
Elements
2 density curves: a dashed light gray curve (standard normal distribution) and a solid black curve (Student's t on 5 df), identified by a legend at upper right; a solid vertical line from the baseline up to the t curve; a dotted gray arrow labeled 1.645 and a solid orange arrow labeled 2.015 point to the right flank of the t curve
Horizontal
-4 to 4, integer ticks; the solid vertical line stands at about 2.0
Vertical
no axis or scale
Visible pattern
The solid t curve is bell-shaped and symmetric about 0, falling to a low, still slightly raised baseline at -4 and 4. The dashed normal curve is very faint; it can be seen just outside the solid curve near the peak, so its peak is a little higher, and is otherwise hard to tell apart. The 2 arrows point to neighboring spots on the right flank, the gray one at about 1.6 and the orange one at about 2.0, where the vertical line meets the curve.

Text in the picture

Student's t distribution on 5 degrees of freedom
1.645
2.015

Slide 67Student's t distribution on 13 degrees of freedom

A Student's t density on 13 degrees of freedom with a standard normal curve in its legend; arrows mark 1.645 and 1.771.
Description
Panels
1
Elements
2 density curves named in a legend at upper right: standard normal distribution (dashed light gray) and Student's t on 13 df (solid black); a solid vertical line from the baseline up to the curve; a dotted gray arrow labeled 1.645 and a solid orange arrow labeled 1.771 point to the right flank of the curve
Horizontal
-4 to 4, integer ticks; the solid vertical line stands at about 1.8
Vertical
no axis or scale
Visible pattern
A single bell-shaped curve is visible, symmetric about 0 and falling to a nearly flat baseline by -4 and 4; the dashed normal curve cannot be told apart from it. The 2 arrows point to nearly the same place on the right flank, the gray one at about 1.6 and the orange one at about 1.8, where the vertical line meets the curve.

Text in the picture

Student's t distribution on 13 degrees of freedom
1.645
1.771

Slide 68Student's t distribution on 21 degrees of freedom

A Student's t density on 21 degrees of freedom with a standard normal curve in its legend; arrows mark 1.645 and 1.721.
Description
Panels
1
Elements
2 density curves named in a legend at upper right: standard normal distribution (dashed light gray) and Student's t on 21 df (solid black); a solid vertical line from the baseline up to the curve; a dotted gray arrow labeled 1.645 and a solid orange arrow labeled 1.721 point to the right flank of the curve
Horizontal
-4 to 4, integer ticks; the solid vertical line stands at about 1.7
Vertical
no axis or scale
Visible pattern
A single bell-shaped curve is visible, symmetric about 0 and falling to a nearly flat baseline by -4 and 4; the dashed normal curve cannot be told apart from it. The 2 arrows point to almost the same place on the right flank, at about 1.6 and 1.7, the orange one meeting the curve where the vertical line stands.

Text in the picture

Student's t distribution on 21 degrees of freedom
1.645
1.721

Slide 69Student's t distribution on 29 degrees of freedom

A Student's t density on 29 degrees of freedom with a standard normal curve in its legend; arrows mark 1.645 and 1.699.
Description
Panels
1
Elements
2 density curves named in a legend at upper right: standard normal distribution (dashed light gray) and Student's t on 29 df (solid black); a solid vertical line from the baseline up to the curve; a dotted gray arrow labeled 1.645 and a solid orange arrow labeled 1.699 point to the right flank of the curve
Horizontal
-4 to 4, integer ticks; the solid vertical line stands at about 1.7
Vertical
no axis or scale
Visible pattern
A single bell-shaped curve is visible, symmetric about 0 and falling to a nearly flat baseline by -4 and 4; the dashed normal curve cannot be told apart from it. The 2 arrows point to almost the same place on the right flank, at about 1.6 and 1.7, the orange one meeting the curve where the vertical line stands.

Text in the picture

Student's t distribution on 29 degrees of freedom
1.645
1.699

Slide 70Specifying critical values

Slide 70: Specifying critical values; its text follows
  • Critical value tk,p is value of standard t distribution on k degrees of freedom such that Pr(T ≤ tk,p) = p
  • Examples:
t5,0.025=–2.571
t13,0.050=–1.771
t21,0.950=1.721
t29,0.975=2.045

Slide 71Compare mean to fixed standard [known variance]

Slide 71: Compare mean to fixed standard [known variance]
  • H0: μ=μ0; H1: μ≠μ0
    • n observations
    • Population variance known, σ2,
  • or n “large”
    • Not too non-normal (depends on n)
  • Test statistic:
  • For 2-sided test, compare absolute value of Z to z1–α/2
Description

Text in the picture

Z = (X-bar − μ_0) / (σ / sqrt(n))

Slide 72Ex: one-sample Z test

Slide 72: Ex: one-sample Z test; its text follows
  • BMI in HIV+ women in HERS
  • H0: μ=25.1; H1: μ≠25.1
  • n=617
  • Critical value z0.975=1.960
  • Sample mean: 25.711
  • Test statistic: 2.42, P=0.016
  • Conclusion:
  • Reject H0 at α=0.05

Abbreviations: BMI = body mass index (kg/m²); HER/HERS = HIV Epidemiology Research (HER) Study; HIV+ = HIV-positive.

Slide 73Compare mean to fixed standard [estimated variance]

Slide 73: Compare mean to fixed standard [estimated variance]
  • H0: μ=μ0; H1: μ≠μ0
    • n<30 observations
    • Population variance unknown
    • Normal (or very nearly so)
  • Test statistic:
  • where s estimates population standard deviation
  • For 2-sided test, compare absolute value of T to tn-1,1–α/2
Description

Text in the picture

T = (X-bar − μ0) / (s / sqrt(n))

Slide 74Ex: one-sample t test

Slide 74: Ex: one-sample t test; its text follows
  • BMI in HERS women of “other” race/ethnicity
  • H0: μ=25.1; H1: μ≠25.1
  • n=11
  • Critical value t10,0.975=2.228
  • Sample mean: 25.785
  • Test statistic: 0.434, P=0.67
  • Conclusion:
  • Do not reject H0 at α=0.05

Abbreviations: BMI = body mass index (kg/m²); HER/HERS = HIV Epidemiology Research (HER) Study.

Slide 75Compare means from 2 independent populations

Slide 75: Compare means from 2 independent populations
  • H0: μ1=μ2; H1: μ1≠μ2
  • Sample sizes n1, n2
  • Test statistic: standardized difference of means
    • n1+n2 > 31: Z
    • n1+n2 < 32, σ12=σ22 unknown:
  • t on n1+n2–2 df
    • n1+n2 < 32, σ12≠σ22 unknown:
  • t on complicated df
Description

Text in the picture

X-bar − Y-bar

Presenter notes

Use t test if n1+n2<32; the degrees of freedom for the t test are more complicated if population variances are unequal

Slide 76Ex: means from 2 independent populations

Slide 76: Ex: means from 2 independent populations; its text follows
  • BMI in HIV+ vs HIV– women
  • H0: μ1=μ2; H1: μ1≠μ2
  • n1=617, n2=318
  • Critical value z0.975=1.960
  • Sample means: 25.711, 26.707
  • Test statistic: 2.285, P=0.022
  • Conclusion:
  • Reject H0 at α=0.05

Abbreviations: BMI = body mass index (kg/m²); HIV = human immunodeficiency virus; HIV+ = HIV-positive.

Slide 77Compare means from paired populations

Slide 77: Compare means from paired populations
  • For each single observation Xi, there is a matched observation Yi
  • Interested in differences Di=Xi–Yi
  • H0: μ1=μ2; H1: μ1≠μ2
  • n pairs
  • Test statistic: standardize
    • Z (n>29)
    • t test on n–1 df (n<30, population variance of D unknown)
Description

Text in the picture

D-bar

Slide 78Ex: means from 2 paired populations

Slide 78: Ex: means from 2 paired populations; its text follows
  • BMI in HIV+ women at enrollment and 12 months later
  • H0: μ1=μ2; H1: μ1≠μ2
  • n=617
  • Critical value z0.975=1.960
  • Sample means: 25.711, 25.627
  • Test statistic: 0.784, P=0.43
  • Conclusion:
  • Do not reject H0 at α=0.05

Abbreviations: BMI = body mass index (kg/m²); HIV+ = HIV-positive.

Slide 79Tests for tables

Slide 79: Tests for tables; its text follows
  • Cross-classify 2+ discrete variables
    • 2 x 2: Gender by HIV status
    • 3 x 3: Political party (D, R, O) by BMI status (<25, 25−<30, ≥30)
    • 2 x 2: Exposure in case crossed with exposure in matched control
  • Test independence of variables

Abbreviations: BMI = body mass index (kg/m²); HIV = human immunodeficiency virus.

Slide 80Chi-square distribution

Slide 80: Chi-square distribution; its text follows
  • Positive values, long right tail
  • Indexed by degrees of freedom
  • Mean of χ2k is k
  • Typically use upper tail only
χ21,0.950=3.84
χ24,0.950=9.49
χ26,0.950=12.59
χ29,0.950=16.92

Slide 81Chi-square distribution on 1 degree of freedom

Chart of a single chi-square curve on 1 degree of freedom, with 3.84 marked below the curve.
Description
Panels
1
Elements
1 black curve, labeled chi-square with subscript 1 by an orange label and arrow; the value 3.84 in orange below the curve, with a faint short vertical line above it
Horizontal
0 to 25, ticks and labels every 1; the curve spans 0 to about 9
Vertical
No axis, scale or labels; the curve is cut off flat at the top of the plotting area
Visible pattern
The curve rises almost vertically at 0 to the top of the plotting area, runs flat there for a short stretch to about 0.4, then falls steeply. By about 4 it lies close to the baseline, and it trails off nearly flat to its end near 9. The rest of the axis, to 25, is empty.

Text in the picture

Chi-square distribution on 1 degree of freedom

Slide 82Chi-square distribution on 4 degree of freedom

Chart of chi-square curves on 1 and 4 degrees of freedom, the 4 curve emphasized, with 3.84 and 9.49 marked.
Description
Panels
1
Elements
2 curves: chi-square with subscript 1 as a thin light gray line with a gray label and arrow, chi-square with subscript 4 as a black line with an orange label and arrow; the values 3.84 (gray) and 9.49 (orange) below the curves, each with a faint short vertical line above it
Horizontal
0 to 25, ticks and labels every 1; the curves span 0 to about 16
Vertical
No axis, scale or labels; the gray curve is cut off flat at the top of the plotting area
Visible pattern
The gray curve keeps the shape of the previous chart: a near-vertical spike at 0, cut off at the top, falling steeply to the baseline. The black curve starts at 0, rises to a single rounded peak near 2 at about a third of the plotting height, and falls away in a long right tail that reaches the baseline near 16. The 2 curves cross near 1.3.

Text in the picture

Chi-square distribution on 4 degrees of freedom

Slide 83Chi-square distribution on 6 degree of freedom

Chart of chi-square curves on 1, 4 and 6 degrees of freedom, the 6 curve emphasized, with 3.84, 9.49 and 12.59 marked.
Description
Panels
1
Elements
3 curves: chi-square with subscripts 1 and 4 as thin light gray lines with gray labels and arrows, chi-square with subscript 6 as a black line with an orange label and arrow; the values 3.84 and 9.49 (gray) and 12.59 (orange) below the curves, each with a faint short vertical line above it
Horizontal
0 to 25, ticks and labels every 1; the curves span 0 to about 20
Vertical
No axis, scale or labels; the 1 curve is cut off flat at the top of the plotting area
Visible pattern
The 2 gray curves repeat the earlier charts. The black curve leaves 0 flat, rises to a rounded peak near 4 at about a quarter of the plotting height, lower than the gray 4 curve's peak, and declines in a long right tail to the baseline near 20. Each successive curve peaks further right, lower and wider.

Text in the picture

Chi-square distribution on 6 degrees of freedom

Slide 84Chi-square distribution on 9 degree of freedom

Chart of chi-square curves on 1, 4, 6 and 9 degrees of freedom, the 9 curve emphasized, with 4 values marked.
Description
Panels
1
Elements
4 curves: chi-square with subscripts 1, 4 and 6 as thin light gray lines with gray labels and arrows, chi-square with subscript 9 as a black line with an orange label and arrow; the values 3.84, 9.49 and 12.59 (gray) and 16.92 (orange) below the curves, each with a faint short vertical line above it
Horizontal
0 to 25, ticks and labels every 1; the black curve runs to the right end of the axis
Vertical
No axis, scale or labels; the 1 curve is cut off flat at the top of the plotting area
Visible pattern
The 3 gray curves repeat the earlier charts. The black curve stays near the baseline at first, rises to a broad rounded peak near 7 at about a fifth of the plotting height, the lowest of the 4 peaks, and declines slowly to near the baseline at 25. Across the 4 curves the peaks step rightward and downward, and the curves spread wider; the marked values step rightward with them.

Text in the picture

Chi-square distribution on 9 degrees of freedom

Slide 852 x 2 table

Slide 85: 2 x 2 table; its text follows
Condition B presentCondition B absent
Condition A presentn11n12n11 + n12
Condition A absentn21n22n21 + n22
n11 + n21n12 + n22n

Slide 86Independence in a 2 x 2 table

Slide 86: Independence in a  2 x 2 table
  • Compute “expected” cell counts based on observed row and column totals
  • Test statistic compares observed and expected cell counts:
  • Compare to χ21 distribution
Description

Text in the picture

(n11 − E11)^2 / E11 + (n12 − E12)^2 / E12 + (n21 − E21)^2 / E21 + (n22 − E22)^2 / E22

Presenter notes

Odds ratio, prevalence ratio

Slide 87Ex: independence in a 2 x 2 table

Slide 87: Ex: independence in a  2 x 2 table; its text follows
  • Overweight (BMI≥25.0) by
  • HIV status in HERS women
  • H0: no association
  • n=935
  • Critical value χ21,0.950 =3.84
  • Test statistic: 5.63, P=0.018
  • Conclusion:
  • Reject H0 at α=0.05

Abbreviations: BMI = body mass index (kg/m²); HER/HERS = HIV Epidemiology Research (HER) Study; HIV = human immunodeficiency virus.

Slide 88Independence in a R x C table

Slide 88: Independence in a  R x C table; its text follows
  • Any number of rows and columns
  • Same idea as 2 x 2 table
    • Observed cell counts
    • Expected cell counts
  • under independence
    • Sum of weighted, squared differences
  • χ2 on (R–1)(C–1) df
Presenter notes

Correction available if expected counts of some cells are too small

Slide 89Ex: independence in a R x C table

Slide 89: Ex: independence in a  R x C table; its text follows
  • Weight category (<25, 25−<30, ≥30) by race/ethnicity (B, L, W, O) in HIV+ women at enrollment
  • H0: no association
  • n=617
  • Critical value χ26,0.950 =12.59
  • Test statistic: 22.81, P<0.001
  • Conclusion:
  • Reject H0 at α=0.05

Abbreviations: HIV+ = HIV-positive.

Slide 90Exact tests for tables

Slide 90: Exact tests for tables; its text follows
  • Use discrete math theory to count all possible tables with same row and column sums as observed
  • P-value: fraction of all tables that favor alternative hypothesis
    • Ex: odds ratio > 1
  • 2 x 2: Fisher’s exact test
  • R x C: Freeman-Halton test
Presenter notes

Often highly discrete

Slide 91Ex: exact tests for tables

Slide 91: Ex: exact tests for tables; its text follows
  • Weight category (<25, 25−<30, ≥30) by race/ethnicity (B, W, H, O) in HIV+ women at enrollment
  • H0: no association
  • n=617
  • P<0.001 (computation took >9 min)
  • Conclusion:
  • Reject H0 at α=0.05
Presenter notes

Often highly discrete

Abbreviations: HIV+ = HIV-positive.

Slide 922 x 2 table with paired data (1)

A 2 x 2 grid of cell labels n11, n12, n21, n22, with the 2 off-diagonal cells (n12 and n21) shaded green.
  • Observe 2 matched binary variables
  • Cross-tabulate
  • Test equality of off-diagonal proportions (discordant pairs)
n11n12
n21n22

Slide 932 x 2 table with paired data (2)

Slide 93: 2 x 2 table with paired data (2); its text follows
  • McNemar’s test for symmetry
    • Extends to larger square tables
    • Continuity correction or exact version for tables with small cells
  • χ2 on 1 df in 2 x 2 tables

Slide 94Ex: 2 x 2 table with paired data

Slide 94: Ex: 2 x 2 table with paired data; its text follows
  • BMI≥25.0 in HIV+ women at enrollment at 12 months later
  • H0: π12=π21; H1: π12≠π21
  • n=617
  • Critical value χ21,0.950 =3.84
  • Test statistic: 0.545, P=0.46
  • Conclusion:
  • Do not reject H0 at α=0.05
29830
36253
Presenter notes

Continuity correction: TS: 0.379, P=0.54
Exact: P=0.54

Abbreviations: BMI = body mass index (kg/m²); HIV+ = HIV-positive; TS = test statistic.

Slide 95Tests for proportions

Slide 95: Tests for proportions; its text follows
  • One sample: H0: π=π0; H1: π≠π0
    • Small samples—exact methods
    • Large samples—Z test
  • Multiple samples
    • Methods for tables

Slide 96Ex: test for proportions

Slide 96: Ex: test for proportions; its text follows
  • Prevalence of BMI≥30.0 in
  • HIV+ women is 25%
  • H0: π=0.25; H1: π≠0.25
  • n=617
  • Critical value z0.975=1.960
  • Sample proportion: 0.2156
  • Test statistic: 1.976, P=0.048
  • Conclusion:
  • Reject H0 at α=0.05

Abbreviations: BMI = body mass index (kg/m²); HIV+ = HIV-positive.

Slide 97Nonparametric tests

Slide 97: Nonparametric tests; its text follows
  • Minimal assumptions about population distributions
  • Typically based on ranks of data
  • Examples
    • Sign test for median
    • Wilcoxon rank-sum test (Mann-Whitney)
    • Wilcoxon signed-rank test for median or paired data
Presenter notes

“distribution-free” “rank-based”

Slide 98Rank test, illustrated (1)

Rug plot of 2 samples of values on a common 0 to 75 scale, one row of red tick marks and one row of blue.
Description
Panels
1, unframed, with one labeled band of 2 rows
Elements
Vertical tick marks (a rug) depict individual values, red for Sample 1 (upper row) and blue for Sample 2 (lower row), with a 2-line color legend at upper right
Horizontal
0 to 75, labeled every 5, axis repeated at top and bottom
Vertical
No scale; a single band labeled Original data near the top
Visible pattern
The red marks run from about 5 to about 52, most densely between about 5 and 31, with one isolated mark near 71. The blue marks begin later, at about 15, and run to about 55, with 2 isolated marks near 66 and 75. The 2 rows overlap across most of the middle range, but the red row extends further to the left and the blue row further to the right.

Text in the picture

Sample 1
Sample 2
Original data

Slide 99Rank test, illustrated (2)

The same 2-sample rug plot, now with a second band showing each value replaced by its rank in the combined sample.
Description
Panels
1, unframed, with 2 labeled bands of 2 rows each
Elements
Vertical tick marks depict individual values (upper band) and their ranks (lower band), red for Sample 1 (upper row of each band) and blue for Sample 2 (lower row), with a 2-line color legend at upper right
Horizontal
0 to 75, labeled every 5, axis repeated at top and bottom, serving as the value scale for the upper band and the rank scale for the lower band; ranks run from 1 to 75
Vertical
No scale; bands labeled Original data and Ranks (in joint sample), stacked top to bottom
Visible pattern
The upper band repeats the original values: red from about 5 to 52 with an outlier near 71, blue from about 15 to 55 with outliers near 66 and 75. In the rank band the marks fall on evenly spaced integer positions. Red holds an unbroken run of the lowest ranks, about 1 to 11, and blue begins at about 12; above that the colors interleave in short runs, with blue holding most of the highest ranks, about 69 to 75, apart from one red mark near 74. Overall the red ranks lean toward the left and the blue ranks toward the right.

Text in the picture

Sample 1
Sample 2
Original data
Ranks (in joint sample)

Slide 100Rank test, illustrated (3)

The 2-sample rug plot with a third band merging both samples' ranks into one row, annotated with a Wilcoxon rank-sum P-value.
Description
Panels
1, unframed, with 3 labeled bands
Elements
Vertical tick marks depict individual values (top band) and ranks (middle and bottom bands), red for Sample 1 and blue for Sample 2; the top 2 bands keep the samples in separate rows, the bottom band merges them into a single row with blue marks set slightly lower than red; a 2-line color legend at upper right
Horizontal
0 to 75, labeled every 5, axis repeated at top and bottom, serving as the value scale for the top band and the rank scale for the lower 2; ranks run from 1 to 75
Vertical
No scale; bands labeled Original data, Ranks (in joint sample), and Ranks (in joint sample) again, stacked top to bottom; the P-value annotation sits at the right above the bottom band
Visible pattern
The top 2 bands repeat the previous slide. In the bottom band all 75 ranks form one continuous evenly spaced row in which the colors alternate: a solid run of red from rank 1 to about 11, then a mix of short red and blue runs across the middle, and mostly blue at the far right, about 69 to 75, with one red mark near 74. Red marks are more frequent in the left half of the row and blue marks in the right half.

Text in the picture

Sample 1
Sample 2
Original data
Ranks (in joint sample)
Ranks (in joint sample)
Wilcoxon rank-sum test P=0.059

Slide 101Some other tests

Slide 101: Some other tests; its text follows
  • Cochran-Mantel-Haenszel test
    • Compare multiple 2-way tables,
  • especially 2 x 2 for interactions
  • Cochran-Armitage test
    • Linear trend in tabulated proportions
  • Shapiro-Wilk test
    • Assess appropriateness of normal distribution

Slide 102[quotations]

Slide 102: [quotations]; its text follows
  • Far too often the choices reality proposes are such as to take away one’s taste for choosing.
  • — Jean Rostand
  • Variety, multiplicity are the two most powerful vehicles of lust.
  • — Marquis de Sade
Presenter notes

Jean Rostand (1894–1977), French biologist, writer. Pensées d’un Biologiste, ch. 10, “A Biologist’s Thoughts” (1939; repr. in The Substance of Man, 1962).
Marquis de Sade (1740–1814), French author. Mme Delbène, in L’Histoire de Juliette, ou les Prospérités du Vice, pt. 1 (1797).

Slide 103Part 3: Hypothesis testing in context Practical and ethical considerations

Slide 103: Part 3: Hypothesis testing in context Practical and ethical considerations; its text follows

Slide 104[quotations]

Slide 104: [quotations]; its text follows
  • The business of the statistician is to catalyze the scientific learning process.
  • — George E P Box
  • Data analysis is an aid to thinking and not a replacement for.
  • — Richard Shillington

Slide 105Practical views on testing

Slide 105: Practical views on testing; its text follows
  • Hypothesis testing is important but limited
  • P-values are more limited, still
  • Practical and ethical considerations

Slide 106Hypothesis testing is important but limited

Slide 106: Hypothesis testing is important but limited; its text follows
  • Role in chain of hypotheses
  • Chance vs non-chance
    • Probability model
    • What is non-chance?
  • Long-term quality control
    • Over time—several samples
    • What about this sample?

Slide 107Role in chain of hypotheses

Slide 107: Role in chain of hypotheses; its text follows
  • Failing to reject H0
    • propagates back to contradicting the substantive hypothesis
  • Rejecting H0
    • does not prove the substantive hypothesis
    • implies that chance is not an explanation for observed phenomena

Slide 108Chance vs non-chance (1)

Slide 108: Chance vs non-chance (1); its text follows
  • Probability model
    • Randomness in sampling
      • Sampling method
      • Test statistic distribution
    • If not identified (does not exist),
  • no statistical test is valid
      • Convenience sample
      • Complete enumeration
      • Systematic sampling deficiencies

Slide 109Chance vs non-chance (2)

Slide 109: Chance vs non-chance (2); its text follows
  • What is “non-chance”?
    • Proper interpretation of rejecting H0: observed difference between test statistic and population parameter was due to factors other than chance
  • Those factors could include
    • H1 is true
    • Deficiencies in model or sampling
      • Uncontrolled confounding

Slide 110Long-term quality control (1)

Slide 110: Long-term quality control (1); its text follows
  • Over time—several samples
    • Hypothesis testing paradigm guarantees a controlled error rate in repeated sampling
    • In 100 properly executed experiments with H0 true and fixed and α=0.05, H0 would be rejected about 5 times
Presenter notes

Independent tests

Slide 111Long-term quality control (2)

Slide 111: Long-term quality control (2); its text follows
  • What about this specific test?
    • There are no guarantees about any given test
    • Error rates can only be controlled in the aggregate
    • Other considerations
      • Rigorous study design, execution
      • Appropriate statistical procedures

Slide 112P-values are more limited

Slide 112: P-values are more limited; its text follows
  • Property of sample, not hypothetical population
    • Random quantity
  • One proper role
  • Significance
  • Inductive validity

Slide 113P-values: one proper role (1)

Slide 113: P-values: one proper role (1); its text follows
  • Only proper inferential role:
  • to allow different researchers their personal confidence level
  • Reject if P < personal α
    • Ex 1: restrictive 0.01 rate
    • Ex 2: conventional 0.05 rate
    • If P = 0.03, the first would not reject, but the second would reject
Presenter notes

Have you hugged your inner α today?

Slide 114P-values: one proper role (2)

Slide 114: P-values: one proper role (2); its text follows
  • What if α=0.05 and P=0.052?
    • Strictly speaking, do not reject H0
    • Possible reasons
      • Approximation of test statistic distribution
      • Discrete distribution
      • Round off error
    • Still, do not reject H0
  • What if α=0.05 and P=0.048?
Presenter notes

Have you hugged your inner α today?

Slide 115P-values: significance

Slide 115: P-values: significance; its text follows
  • Practical significance—non-statistical
  • Not a direct or rigorous measure of strength of evidence
  • Small P does not mean
    • Important result
    • Strong evidence
    • H0 wrong by a large amount
  • Large P does not mean
    • Unimportant result
    • Weak evidence
Presenter notes

For a true measure of evidence, other statistical methods must be used
It’s all about chance

Slide 116P-values: inductive validity

Slide 116: P-values: inductive validity; its text follows
  • Induction by exclusion of alternative explanations
    • Method of difference
      • Only due to chance
      • Uncontrolled confounders?
    • Rigor: conceptual, design, execution
    • Causality?
Presenter notes

Define method of difference
Causality is a non-statistical concept: no test by itself can prove causality

Slide 117Practical and ethical considerations

Slide 117: Practical and ethical considerations; its text follows
  • Multiple tests
  • One-sided vs two-sided tests
  • Ethics of H0 and α
  • Ethics of H1 and β
  • Reporting α, β, and P

Slide 118Multiple tests

Slide 118: Multiple tests; its text follows
  • Chance of at least one error increases quickly
  • Affect α, β, and P
  • Particular problem in observational studies, multiple regression
    • Seeking out “significant” results
  • Adjustment procedures
    • Set stringent rate per test
    • or Upsize P post hoc

Slide 119One-sided vs two-sided

Slide 119: One-sided vs two-sided; its text follows
  • Generally speaking, 2-sided tests are preferred
  • Defending a 1-sided test entails added burden
    • Might want to detect harm but ignore benefit
    • Could be unethical to detect benefit but ignore harm

Slide 120Ethics of H0 and α

Slide 120: Ethics of H0 and α; its text follows
  • If goal is to influence treatment or practice, might set H0 at smallest difference that would motivate changes
    • Ex: H0: θ = θd; H1: θ ≥ θd
  • Choice of α may have ethical import
    • High risk—small α

Slide 121Ethics of H1 and β

Slide 121: Ethics of H1 and β; its text follows
  • High power helps resolve “null” results
    • What if H0 is not rejected?
  • Not the same as no difference
    • Can help rule out specific small differences
  • Human subjects—ethical balance
    • Minimize risk: enroll few as possible
    • Maximize benefit: enroll enough to have power against specified differences

Slide 122Controlling error rates: larger α (lower confidence)

Slide 122: Controlling error rates: larger α (lower confidence); its text follows
  • For fixed sample size and scheme,
  • α and β are inversely related
  • 4 factors that improve power
    • Larger α (lower confidence)

Slide 123Example of beta: alpha 0.05

Two bell-shaped curves centered at 92 and 100, with a dark gray 5% tail under the right curve and a pale yellow 36.1% area under the left curve.
Description
Panels
1
Elements
2 bell-shaped curves of the same width and height, each with a dashed vertical line at its center; a third dashed vertical line at a cutpoint; under the right-hand curve, the area left of the cutpoint is shaded dark gray, with an orange arrow from the label "5%"; under the left-hand curve, the area right of the cutpoint is shaded pale yellow, with an orange arrow from the label "36.1%"; a parameter table is placed at the upper left.
Horizontal
74 to 114, ticks every 2; curves span about 78 to 114; thin dashed verticals at 92 and 100, bolder dashed vertical at about 93
Vertical
No axis or scale; both curves rise from a common baseline to the same peak height, with the parameter table to the left of the left curve's peak
Visible pattern
The 2 curves overlap broadly, one peaked at 92 and the other at 100, crossing at about 96. The bolder dashed line at about 93 divides the shading: dark gray on the left , pale yellow on the right. An arrow labeled 5% points into the gray region, and an arrow labeled 36.1% points into the yellow region.

Text in the picture

Example of beta
SD(pop'n) 24
mean-null 100
mean-alt 92
N 36
SD(mean) 4
alpha 0.05
beta 0.361

Abbreviations: SD = standard deviation.

Slide 124Example of beta: change alpha from 0.05 to 0.10

Two overlapping bell curves with dark gray and pale yellow areas split by a dashed line, and a table of old and new parameter values.
Description
Panels
1
Elements
2 bell-shaped curve outlines of the same width and height; 2 shaded areas under them (dark gray, pale yellow); 3 dashed vertical lines (thin lines at each curve's peak, a bolder line between them); 2 orange arrows with percentage labels pointing into the shaded areas
Horizontal
Unlabeled number line, 74 to 114, ticks every 2; curves span about 78 to 114; thin dashed verticals at 92 and 100, bolder dashed vertical at about 95
Vertical
No axis or scale; both curves rise from a common baseline to the same peak height
Visible pattern
The same 2 overlapping curves, peaked at 92 and 100 and crossing at about 96, with the bolder dashed line now further right, at about 95. The dark gray region under the left tail of the curve peaked at 100 is larger than before, running from about 86 to 95; the pale yellow region under the curve peaked at 92 is smaller, running from about 95 to about 106. The arrow labeled 10% points into the gray region, and the arrow labeled 23.6% points into the yellow region. In the table, the Old column is in lighter gray, the alpha and beta rows are in red (Old values a paler red), and only those 2 rows differ between Old and New.

Text in the picture

Example of beta: Change alpha from 0.05 to 0.10
Old New
SD(pop'n) 24 24
mean-null 100 100
mean-alt 92 92
N 36 36
SD(mean) 4 4
alpha 0.05 0.10
beta 0.361 0.236
10%
23.6%

Abbreviations: SD = standard deviation.

Slide 125Controlling error rates: larger differences between H0 and H1 (effect size)

Slide 125: Controlling error rates: larger differences between H0 and H1 (effect size); its text follows
  • For fixed sample size and scheme,
  • α and β are inversely related
  • 4 factors that improve power
    • Larger α (lower confidence)
    • Larger differences between H0 and H1

Slide 126Example of beta: mean-alt 92

Two bell-shaped curves centered at 92 and 100, with a dark gray 5% tail under the right curve and a pale yellow 36.1% area under the left curve.
Description
Panels
1
Elements
2 bell-shaped curves of the same width and height, each with a dashed vertical line at its center; a third dashed vertical line at a cutpoint; under the right-hand curve, the area left of the cutpoint is shaded dark gray, with an orange arrow from the label "5%"; under the left-hand curve, the area right of the cutpoint is shaded pale yellow, with an orange arrow from the label "36.1%"; a parameter table is placed at the upper left.
Horizontal
74 to 114, ticks every 2; curves span about 78 to 114; thin dashed verticals at 92 and 100, bolder dashed vertical at about 93
Vertical
No axis or scale; both curves rise from a common baseline to the same peak height, with the parameter table to the left of the left curve's peak
Visible pattern
The 2 curves overlap broadly, one peaked at 92 and the other at 100, crossing at about 96. The bolder dashed line at about 93 divides the shading: dark gray on the left , pale yellow on the right. An arrow labeled 5% points into the gray region, and an arrow labeled 36.1% points into the yellow region.

Text in the picture

Example of beta
SD(pop'n) 24
mean-null 100
mean-alt 92
N 36
SD(mean) 4
alpha 0.05
beta 0.361

Abbreviations: SD = standard deviation.

Slide 127Example of beta: change mean-alt from 92 to 88

Two overlapping bell-shaped curves with shaded tails on either side of a dashed cutoff line, beside a table comparing old and new parameter values.
Description
Panels
1
Elements
2 bell-shaped curves (thin black lines) of equal width and height, each with a thin gray dashed vertical line at its peak; 1 black dashed vertical line at a cutoff; 2 shaded areas meeting at the cutoff, one dark gray and one pale yellow; 2 orange arrows with orange percentage labels (5% and 19.6%) pointing into the shaded areas. A parameter table sits at the upper left: Old column in gray, New column in black, with the changed rows (mean-alt and beta) in red, paler red in the Old column. No legend; the curves themselves are unlabeled, and their peaks fall at the New values of mean-alt (88) and mean-null (100) in the table.
Horizontal
74 to 114, labeled ticks every 2; the left curve begins near 74 and the right curve ends near 114; dashed vertical guides at about 88 and 100 (the peaks) and at about 93.4 (the cutoff)
Vertical
no axis or scale; both curves rise to the same height, with the parameter table filling the space above and to the left of the left curve's peak
Visible pattern
The 2 curves are the same shape, set 12 units apart, and overlap between roughly 86 and 102, crossing near 94. The cutoff line stands between the peaks, somewhat closer to the left one. The dark gray area is a thin sliver under the left tail of the right curve, running from about 86 up to the cutoff, where it reaches about a quarter of the peak height; the 5% label points to it. The pale yellow area lies under the right tail of the left curve, starting at the cutoff at a bit over a third of the peak height and tapering to the baseline, with a faint yellow line continuing along the baseline to about 106; the 19.6% label points to it.

Text in the picture

Example of beta: Change mean-alt from 92 to 88
Old New
SD(pop'n) 24 24
mean-null 100 100
mean-alt 92 88
N 36 36
SD(mean) 4 4
alpha 0.05 0.05
beta 0.361 0.196
5%
19.6%

Abbreviations: SD = standard deviation.

Slide 128Controlling error rates: larger sample sizes (precision)

Slide 128: Controlling error rates: larger sample sizes (precision); its text follows
  • For fixed sample size and scheme,
  • α and β are inversely related
  • 4 factors that improve power
    • Larger α (lower confidence)
    • Larger differences between H0 and H1
    • Larger sample sizes

Slide 129Example of beta: N 36

Two bell-shaped curves centered at 92 and 100, with a dark gray 5% tail under the right curve and a pale yellow 36.1% area under the left curve.
Description
Panels
1
Elements
2 bell-shaped curves of the same width and height, each with a dashed vertical line at its center; a third dashed vertical line at a cutpoint; under the right-hand curve, the area left of the cutpoint is shaded dark gray, with an orange arrow from the label "5%"; under the left-hand curve, the area right of the cutpoint is shaded pale yellow, with an orange arrow from the label "36.1%"; a parameter table is placed at the upper left.
Horizontal
74 to 114, ticks every 2; curves span about 78 to 114; thin dashed verticals at 92 and 100, bolder dashed vertical at about 93
Vertical
No axis or scale; both curves rise from a common baseline to the same peak height, with the parameter table to the left of the left curve's peak
Visible pattern
The 2 curves overlap broadly, one peaked at 92 and the other at 100, crossing at about 96. The bolder dashed line at about 93 divides the shading: dark gray on the left , pale yellow on the right. An arrow labeled 5% points into the gray region, and an arrow labeled 36.1% points into the yellow region.

Text in the picture

Example of beta
SD(pop'n) 24
mean-null 100
mean-alt 92
N 36
SD(mean) 4
alpha 0.05
beta 0.361

Abbreviations: SD = standard deviation.

Slide 130Example of beta: change N from 36 to 64

Two narrower, taller overlapping bell-shaped curves with shaded tails on either side of a dashed cutoff line, beside a table comparing old and new values.
Description
Panels
1
Elements
2 bell-shaped curves (thin black lines) of equal width and height, each with a thin gray dashed vertical line at its peak; 1 black dashed vertical line at a cutoff; 2 shaded areas meeting at the cutoff, one dark gray and one pale yellow; 2 orange arrows with orange percentage labels (5% and 15.3%) pointing into the shaded areas. A parameter table sits at the upper left: Old column in gray, New column in black, with the changed rows (N, SD(mean) and beta) in red, paler red in the Old column. No legend; the curves themselves are unlabeled, and their peaks fall at the values of mean-alt (92) and mean-null (100) in the table.
Horizontal
74 to 114, labeled ticks every 2; the curves span about 81 to 110; dashed vertical guides at about 92 and 100 (the peaks) and at about 95 (the cutoff)
Vertical
no axis or scale; both curves rise to the same height, nearly to the level of the table's Old/New header, with the table to the left of the left curve
Visible pattern
Compared with the curves on the neighboring beta slides drawn on the same horizontal scale, these 2 are narrower and rise higher, and they overlap less, crossing near 96 at about a third of the peak height. The cutoff line stands about midway between the peaks, a little left of center. The dark gray area is a narrow sliver under the left tail of the right curve, from about 86 to the cutoff, where it reaches about a quarter of the peak height; the 5% label points to it. The pale yellow area lies under the right side of the left curve beyond the cutoff, starting at a bit under 2-thirds of the peak height and tapering to the baseline near 102, with a faint yellow line continuing along the baseline to about 106; the 15.3% label points to it.

Text in the picture

Example of beta: Change N from 36 to 64
Old New
SD(pop'n) 24 24
mean-null 100 100
mean-alt 92 92
N 36 64
SD(mean) 4 3
alpha 0.05 0.05
beta 0.361 0.153
5%
15.3%

Abbreviations: SD = standard deviation.

Slide 131Controlling error rates: homogeneity of population (variance)

Slide 131: Controlling error rates: homogeneity of population (variance); its text follows
  • For fixed sample size and scheme,
  • α and β are inversely related
  • 4 factors that improve power
    • Larger α (lower confidence)
    • Larger differences between H0 and H1
    • Larger sample sizes
    • Homogeneity of population

Slide 132Example of beta: SD 24

Two bell-shaped curves centered at 92 and 100, with a dark gray 5% tail under the right curve and a pale yellow 36.1% area under the left curve.
Description
Panels
1
Elements
2 bell-shaped curves of the same width and height, each with a dashed vertical line at its center; a third dashed vertical line at a cutpoint; under the right-hand curve, the area left of the cutpoint is shaded dark gray, with an orange arrow from the label "5%"; under the left-hand curve, the area right of the cutpoint is shaded pale yellow, with an orange arrow from the label "36.1%"; a parameter table is placed at the upper left.
Horizontal
74 to 114, ticks every 2; curves span about 78 to 114; thin dashed verticals at 92 and 100, bolder dashed vertical at about 93
Vertical
No axis or scale; both curves rise from a common baseline to the same peak height, with the parameter table to the left of the left curve's peak
Visible pattern
The 2 curves overlap broadly, one peaked at 92 and the other at 100, crossing at about 96. The bolder dashed line at about 93 divides the shading: dark gray on the left , pale yellow on the right. An arrow labeled 5% points into the gray region, and an arrow labeled 36.1% points into the yellow region.

Text in the picture

Example of beta
SD(pop'n) 24
mean-null 100
mean-alt 92
N 36
SD(mean) 4
alpha 0.05
beta 0.361

Abbreviations: SD = standard deviation.

Slide 133Example of beta: change SD from 24 to 18

Two narrower, taller overlapping bell-shaped curves with shaded tails on either side of a dashed cutoff line, beside a table comparing old and new values.
Description
Panels
1
Elements
2 bell-shaped curves (thin black lines) of equal width and height, each with a thin gray dashed vertical line at its peak; 1 black dashed vertical line at a cutoff; 2 shaded areas meeting at the cutoff, one dark gray and one pale yellow; 2 orange arrows with orange percentage labels (5% and 15.3%) pointing into the shaded areas. A parameter table sits at the upper left: Old column in gray, New column in black, with the changed rows (SD(pop'n), SD(mean) and beta) in red, paler red in the Old column. No legend; the curves themselves are unlabeled, and their peaks fall at the values of mean-alt (92) and mean-null (100) in the table.
Horizontal
74 to 114, labeled ticks every 2; the curves span about 81 to 110; dashed vertical guides at about 92 and 100 (the peaks) and at about 95 (the cutoff)
Vertical
no axis or scale; both curves rise to the same height, nearly to the level of the table's Old/New header, with the table to the left of the left curve
Visible pattern
Compared with the curves on the neighboring beta slides drawn on the same horizontal scale, these 2 are narrower and rise higher, and they overlap less, crossing near 96 at about a third of the peak height. The cutoff line stands about midway between the peaks, a little left of center. The dark gray area is a narrow sliver under the left tail of the right curve, from about 86 to the cutoff, where it reaches about a quarter of the peak height; the 5% label points to it. The pale yellow area lies under the right side of the left curve beyond the cutoff, starting at a bit under 2-thirds of the peak height and tapering to the baseline near 102, with a faint yellow line continuing along the baseline to about 106; the 15.3% label points to it.

Text in the picture

Example of beta: Change SD(pop'n) from 24 to 18
Old New
SD(pop'n) 24 18
mean-null 100 100
mean-alt 92 92
N 36 36
SD(mean) 4 3
alpha 0.05 0.05
beta 0.361 0.153
5%
15.3%

Abbreviations: SD = standard deviation.

Slide 134Reporting test design

Slide 134: Reporting test design; its text follows
  • Set α and β during design
    • At least before analysis
    • Post hoc is problematic
  • Report
    • Name of test, software
    • Pre-selected error rates, α and β
  • Put α and β in context
    • Informed by ethics? Prior research?
    • Relate power to sample size

Slide 135Reporting P

Slide 135: Reporting P; its text follows
  • State P
    • Or “significant at α=[x]”
    • Or “non-significant at α=[x]”
  • Avoid conventional modifiers of “significant” at different Ps
    • Highly, moderately, marginally
    • Trend (non-significant)
  • 2 or 3 decimal places
    • Ex: <0.001; 0.001–0.099; 0.10–0.99

Slide 136Alternatives to testing

Slide 136: Alternatives to testing; its text follows
  • Decisions and evidence are different concepts
    • For binary decisions with long-term control of error rates, no real alternative to α, β, and P
    • Methods exist for measuring evidence (likelihood, Bayesian); may not control decision error rates

Slide 137Part 4: Postscript

Slide 137: Part 4: Postscript; its text follows

Slide 138Two confessions

Slide 138: Two confessions; its text follows
  • The HERS sample was not strictly random. How can it be used that way?
  • Population mean BMI among US women was closer to ~25.1
    • Ages 20–74 in 1988–1994
    • Extrapolated unofficially from NHANES 3 data
    • Not 26.2 or 26.0 as represented in examples

Abbreviations: BMI = body mass index (kg/m²); HER/HERS = HIV Epidemiology Research (HER) Study; NHANES = National Health and Nutrition Examination Survey; US = United States.

Slide 139Availability of material

Slide 139: Availability of material; its text follows
  • This material will be made available on the EIS extranet in August
    • Presentation slides (PPT, HTM)
    • Dataset (text, SAS)
    • SAS code to reproduce tests

Abbreviations: EIS = Epidemic Intelligence Service (CDC).

Slide 140Acknowledgements

A redacted rectangular region below the title.

[generalized: acknowledgees named on original slide withheld]

Slide 141A personal word

Slide 141: A personal word; its text follows

Slide 142[quotations]

Slide 142: [quotations]; its text follows
  • Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.
  • — John W Tukey

Abbreviations

BMI = body mass index (kg/m²); CDC = Centers for Disease Control and Prevention; EIS = Epidemic Intelligence Service (CDC); HER/HERS = HIV Epidemiology Research (HER) Study; HHS = Department of Health and Human Services; HIV = human immunodeficiency virus; HIV+ = HIV-positive; NHANES = National Health and Nutrition Examination Survey; ResH = research hypothesis; SD = standard deviation; SE = standard error; StaH = statistical hypothesis; SubH = substantive hypothesis; TS = test statistic; US = United States; USA = United States of America.

References, compiled for this page

Full citations, with current links, for selected references and related works.

Full references: whole site, by domain (opens in a new tab) · whole site, by type of work (opens in a new tab)