goodthingswithdata.com

7  Additional practices: How else might data presentation improve?

This section recommends ways to enrich the resources available to readers outside of published reports, such as additional technical details, supporting data, and analytic code, while further enhancing focus in the report itself.

7.1 Intensive exposition of methods

Principle: Sophisticated, unfamiliar, or unconventional methods should be presented in balance with other detail while supporting focus on the public health issue and impact.

Observations: Some reports have extensive exposition on methods, either in the body of the report or in footnotes. Other reports might benefit from additional details, including reports mm6802a1 (García et al. 2019), mm6935a2 (Moreland et al. 2020), mm7001a4 (Leidner et al. 2021), mm7006e2 (Joo et al. 2021), mm7010e3 (Guy et al. 2021), mm7010e4 (Kompaniyets et al. 2021), mm7032e3 (Moline et al. 2021), and mm7110e1 (Donovan et al. 2022).

Recommendations:

  1. MMWR should allow for technical supplements, just as supplemental tables and figures are supported. In addition, MMWR should develop guidance on how to ensure that reports that warrant technical supplementation maintain proper focus and scope within the body of the report. Alternatively, MMWR could develop guidance for authors to place methods in a technical report posted to a public resource besides CDC Stacks, such as prepublication archives.

7.2 Incorporate less familiar methods

Principle: Public health practice can and should gather information and learn through the full variety of available data types and analytic methods, whether familiar or unfamiliar. Examples of ascendent sources include electronic health records and wastewater. Methods with potential to enrich public health findings within the MMWR include, for example, economic methods and prevention effectiveness as well as machine learning, especially as applied to textual and image-based data. In all cases, data uses in MMWR should be rigorous and focus on public health relevance.

Observations:

  1. Several reports analyze data from electronic health record services or vendors, including athenahealth in report mm6802a1 (García et al. 2019) and Cerner in report mm7212e1 (Chun et al. 2023), as well as systems fed by electronic health records or laboratory records as part of networks or programs (such as PCORnet and the National Syndromic Surveillance Program), state systems, and hospital systems.
  2. Report mm6935a2 (Moreland et al. 2020) analyzed anonymized location data from mobile devices.
  3. None of the reports reviewed here include economic analysis, but a few published reports do, including mm6646a3 (Marx et al. 2017), mm7015a1 (Luo et al. 2021), mm7048a1 (Peterson, Miller, et al. 2021), and mm7048a2 (Peterson, Luo, et al. 2021).
  4. None of the reports reviewed here include machine learning as applied to text or image data, but a few published reports mention machine learning, as in report mm7044e1 (Bozio et al. 2021), or cite references pertaining to machine learning, as in report mm7049a3 (Miller et al. 2021).

Recommendations: Stimulate and cultivate a richer set of methods, perhaps through focused solicitation with guest editors, keeping tight focus on public health relevance and credibility.

7.3 Open data and analytic code

Principles: The Executive Branch has encouraged making data publicly available since 2013 and has signaled that federally supported publications will also be expected to make associated data and analytic code available in tandem with peer-reviewed publications. In addition, academia, government, and even the commercial sector have been advancing access to analytic code through public repositories, and many peer-reviewed journals require a statement regarding availability. (OSTP 2022; White House Office of Science and Technology Policy 2022)

Observation: While many reports cite publicly available data sources used within those respective reports, there is no current practice for sharing or describing availability of data or analytic code.

Recommendations:

  1. Require a brief statement on availability of data and analytic code, including whether items are unavailable, available with restriction, or available without restriction.
  2. Develop boilerplate text to describe common modes of availability.
  3. Allow and encourage publicly available data, or data available through a specific, stable request process; data might be limited to machine-readable elements sufficient to reconstruct published and supplemental tables and figures.
  4. Allow and encourage publicly available analytic code, or code available through a specific, stable request process, usually posted on a repository like GitHub.