goodthingswithdata.com

Data in the MMWR

Interpretation (9 of 12)

Data in the MMWR (§6.1): Interpret effects and their significance
Posted to LinkedIn

2026-02-10

NotePreviously published

Posted to LinkedIn on 2026-02-10. The text below is the post as published.

Read the original on LinkedIn

Links to works now on this site point to their pages here; the words are as published. Where the post showed a web address, the icon stands in for it (hover over it to see the address as posted).

Data in the MMWR (§6.1): Interpret effects and their significance On this site (opens in a new tab)
[2026-02-10]

Principles: Measured effects of interest should be described in terms of their public health (or clinical) significance and the variability or uncertainty that comes from using observed data. What is the clinical or public health significance of a measured or derived magnitude of an effect? Interpretation of an effect must match the method(s) used to measure or derive the effect. How compatible is the observed effect size with a reference hypothesis about the world, taking into account the variability or uncertainty that comes with data? Describe sources of uncertainty, including random variation and nonrandom influences. Statistical significance, if presented at all, should be used only to assess the compatibility of results with a background hypothesis, mindful of additional assumptions, potential biases, and variability. One can say that the data are more or less compatible with the hypothesis of interest, not whether the hypothesis itself is true or false.

Observations: One report relies exclusively on statistically significant differences above a population threshold to identify groups above that threshold. The report states but does not further interpret the magnitude of the highest values. Two reports present associations between mask-mandate policies and aggregated incidence values. Although the regression model that is used in both reports is attested in the literature, the analytic method does not admit a straightforward interpretation from which public health implications can be inferred. One report models the probability of 4 possible outcomes with fractional polynomials to pick up nonlinear associations with body mass index (BMI). Although sound, the report overinterprets the specificity of the functional form and the estimates for BMI values with the least risk. One report analyzes data with follow-up that varies in duration between 30 and 365 days, yielding cumulative incidence. The analysis frames results as “absolute risk difference”, which is not supported without additional assumptions.

Recommendations: Reports should interpret the public health significance of the main reported values. Exceptions should be justified, with an explicit note that the public health significance remains to be determined. Qualify whether a statement comprises a comparison of observed magnitude or a formal inferential procedure. P-values and CIs should be interpreted in the context of public health significance. It is insufficient to leave all inferential procedures to be interpreted on their own.

Next topic: Describe potential impact to the extent possible (§6.2)

Data in the MMWR