Data modernization is people: a progressive culture for data in public health

A progressive culture for data in public health

Young Professionals Network 9th Annual Conference, 2022-06-28

Chad Heilig, PhD • [redacted: email address]
Associate Director for Data Science, CSELS

• Data modernization (small em) is people
  • The Data Modernization (large em) Initiative and its antecedents
  • A progressive culture for data in public health
• How CDC should think about data science
  • Data science: why, what, how, who
    • People who do data science (practitioners, experts; learners, mentors)
    • People who empower data science (experts, managers, laypersons; mentors, advocates)
• Data modernization (small em) is people
  • Culture, workforce, leadership, especially early career

• BA in mathematics, PhD in statistics
  • Plus humanities, especially philosophy
• At CDC since December 1997 (as contractor) and February 2000 (as federal employee)
  • Reproductive health (cohorts, clinical trials); human research protections; tuberculosis (clinical trials, screening rules)
• Associate Director for Data Science in CSELS, under DDPHSS, since January 2015

• 2013-2018 Surveillance Strategy
• Notifiable diseases, syndromic surveillance, laboratory records, mortality records
• Surveillance Data Platform
• 2018-2019 Public Health Data Strategy, IT Transformation →
• 2019-2022 Data Modernization Initiative
• eCR, appropriations, Covid-19, appropriations, forecasting, strategic implementation plan

• Data have value because they help us learn things about the world and do things based on those learnings
• Keep up with fast-moving methods, tools, and technology
  • For continuously learning things about the world
  • For empowering choices informed by those learnings
• Public health scientists who care about data can flourish in a culture that
  • Fosters technical skills
  • Inspires and rewards intellectual drive
  • Supports learners in community
• This progressive culture for data complements modernization

• Classic issues:
• fundamentals
• Life cycle of data
• Data sources
  • Experimental
  • Observational
  • Incidental
• Conventional or familiar methods
  • Statistics
  • Causal inference

• Contemporary issues:
• openness and scale
• Open data, privacy
• Reproducibility, traceability
• High-volume data
• High-velocity data
• Nontraditional sources and structures for data
• (EHRs; text, images)
• Less familiar methods

• Data science is about learning from data
  • Bring together, in a rigorous way, all that goes into doing good things with data, in a way that other disciplines do not
  • Keep up with fast-moving methods, tools, and technology for learning from data of all speeds, sizes, structures, and shapes
• Data science is as much about small, static data as it is about complex or dynamic data

• From data, we learn things about the world and inform choices to interact with the world.
• About data in many shapes, structures, sizes, and speeds, varied characteristics that enrich and constrain the ways that data reveal characteristics of the world.
• With data through its full life cycle, posing, answering, and communicating results from rich questions about the world.

• Data science: Extraction of actionable knowledge directly from data through a process of discovery, or hypothesis formulation and hypothesis testing
• Data scientist: Practitioner
  • with sufficient knowledge in the overlapping regimes of business needs, domain knowledge, analytical skills, and software and systems engineering
  • to manage the end-to-end data processes in the data life cycle
• End-to-end role ensures that everything is performed correctly to explore the data, create and validate hypotheses

• Data science, in other words,
  • Studies how to learn from data,
• especially (but not exclusively) complex or nontraditional data
  • Combines analytic, computational, and subject-matter methods to connect the whole life cycle of data

• Critical reflection and iteration
• Set expectations about each core activity
• Collect information about expectations for activity
• Compare expectations to collected information
  • • Mismatch → revise expectations or data
    • Match → cheers!

• Core activities
• State a good question
• Obtain and explore data
• Build a formal model
• Interpret results and implications
• Communicate methods, results, and significance
• Make the life cycle transparent and enduring*

Adapted from Peng R, Matsui E. The Art of Data Science. leanpub.com/artofdatascience. *I added the last item.

• Life cycle. We learn from (and about and with) data in the context of an overall life cycle.
• Analysis centrally connects the life cycle of data.
  • For example, if we know of methods for analyzing images, then we can ask questions that only images can answer
  • Analysis includes statistics, causal inference, machine learning, geospatial methods, econom(etr)ics, and a rich variety of other approaches
• Scientific norms. We judge data science approaches and claims by scientific norms.
  • Example norms: out-of-sample generalizability, traceability, fairness (re: bias), model robustness and stability
  • Science leads, technology follows

• Big data (nor volume, velocity, variety, …)
  • Dynamic and complex technologies and data motivate data science, which aims to keep up with complexity, but do not define data science
• Statistics, machine learning, other data-analytic approaches
  • Data science centers on analysis for principled thinking about using data to answer questions, but it's not the whole life cycle
• Informatics
  • The how and why of technology and systems, not geared to learning from data
  • Informatics undergirds and overlaps some aspects of doing data science
• Just good science
  • Good science adheres to the third commitment of data science (scientific norms) but not necessarily the other 2 ([1] life cycle [2] centered on analysis)

• At the individual level, CDC should foster
  • Technical skills to conduct core activities of the data life cycle
  • Nontechnical skills for critical reflection and iteration through the life cycle
• At the collective level, CDC should foster
  • Progressive culture that supports learners and doers using skills to do good things with data
  • Community and leadership

• Core activities
• State a good question
• Obtain and explore data
• Build a formal model
• Interpret results and implications
• Communicate methods, results, and significance
• Make the life cycle transparent and enduring*

• Technical skills
• Domain knowledge, such as epidemiology
• Computation, e.g., data structures and programming
• Data analysis, e.g., statistics and machine learning
• Verbal and visual communication, e.g., plain language, tables, graphics

Adapted from Peng R, Matsui E. The Art of Data Science. leanpub.com/artofdatascience. *I added the last item.

• Critical reflection and iteration
• Set expectations about activity
• Collect information about expectations
• Compare expectations to information
  • • Mismatch → revise expectations or data
    • Match → cheers!

• Nontechnical skills
• Skills of a good learner
  • Start learning, head in the right direction
  • Keep learning process on track
  • Deal with obstacles
• Skills of a leader

Adapted from Peng R, Matsui E. The Art of Data Science. leanpub.com/artofdatascience.

• Traits that flow from a love of knowledge and learning
  • Ability or skill specific to a virtue, leading to action
  • Motivation or commitment to apply the virtue
  • Judgment or sensitivity when and how to exercise the skill
  • — Baehr 2015
• Too much or too little → vice
• Intellectual virtues animate (data) science

It’s curiosity

• Intellectual traits can run into trouble
  • Too much or too little of a trait (vice)
  • Conflict between virtues (e.g., courage and humility)
• Practical wisdom mediates vice and conflict
  • Grasp which intellectual activity is most valuable for attaining one’s goals
  • Judgment or sensitivity regarding when and when not to exercise that virtue—a dimension of all intellectual virtues

• Learning
  • Formal or informal, structured or unstructured
• Doing
  • Practitioners and those who support or empower them
• Staffing (not discussed further here)
  • Recruit, retain, organize, develop; federal and nonfederal
• Leading
  • Regardless of career stage, job title or series, credential, or location in the hierarchy (some constraints on nonfederal staff)

• Credo for a progressive culture for data
  • Everyone who wants to do good things with data should have the intellectual support to do so.
• Self-learning problem solvers get to do data science
  • Regardless of their area of discipline or expertise
  • Duty to proceed with rigor and stand behind their work

• Guide technical skills
  • Domain skills, computation, data analysis
  • Encourage personalized direction for self-learning
• Model intellectual virtues and exercise practical wisdom
  • Show by example ways to practice curiosity, courage, humility
  • Thinking out loud is an exercise in vulnerability
• Create opportunities to practice intellectual virtues
  • Stimulate curiosity; reward courage; foster humility
  • Cultivate practical wisdom, judgment, sensitivity, discernment

• Respond to and nurture a love of and interest in
• thinking, learning, and answering good questions
• Personal: Mentorees’ beliefs about knowledge and inquiry matter to how one learns about and from data
• Relational: Personal growth in trusting and caring relationships
• Reflective: Pause to discuss the value of thinking
• and the significance of what is being learned

— Baehr 2015

• Direct mentoring of learners
  • Fellows
  • Students
  • Early-career professionals
• Communities of peers
• Meta-mentoring of other mentors

Mentor

Meta-mentor

Learner

Learner

• Take responsibility for self-learning
• Seek mentorship; mentor peers
• Practice intellectual virtues
• Take risks, make interesting mistakes
• Strive to
  • Ask questions
  • Solve problems
  • Think about process
  • Use data [corrected: responsibly] to improve the world

• Influence the practice and profession of (data) science
  • Promote and reward commitment to data, self-learning
  • Remove needless barriers (e.g., technology-related procedures)
  • Support instructive failures and interesting mistakes
• Encourage and uphold learners and doers
• Uphold those who profess data science
• Managers, decision-makers, associate directors

• Practitioners
• Basic or intermediate data skills
• Come from any discipline to do good things with data
• Learn continuously and show how modern tools and methods solve modern problems
• Experts
• Go deep on data science methods
• Set norms for the practice of data science
• Enable, guide, correct, empower practitioners to proceed with rigor and stand behind their work

• Managers
• Ensure resources and direction to achieve good things with data
• Advocate for increasing capability, efficiency, and effectiveness
• Hold to account for knowledge learned from, about, and with data
• Laypersons
• Work with practitioners, experts, and managers
• Assess, use, and advocate for learning from data
• Help ensure supportive resources to enable learning

• Includes practitioners, experts, managers, and laypersons
• Regardless of career stage, job title, credential, or job series
• Leadership continually shapes and sustains the culture of good data practice
• Governance enables effective leadership; governance does not substitute for leadership

• Seek learning, especially self-learning with mentors and peers (be a peer and a mentor!)
• Co-lead
  • Put the modernization in DMI by claiming a role as co-leader
  • Promote leadership, empowerment, and wise innovation
• (in addition to gatekeeping)
• Say what you want and need as modernizers
  • Use your voices to assert what you and your peers want and need for joining or staying in the public health workforce
  • (Previous workforce assessments have focused on managers)

is people!!!!!

Chad Heilig, PhD • [redacted: email address]
Associate Director for Data Science

• Data have value because they help us learn things about the world and do things based on those learnings
• Center on learning and doing
  • Keeping up, not just catching up, on fast-moving methods, tools, and technology
  • People and culture, not just workforce
  • Learners and mentors, not just trainees
  • Leaders from all levels, not just managers or governance

Adapted from Peng R, Matsui E. The Art of Data Science. leanpub.com/artofdatascience

• Start learning, head in the right direction
  • Curiosity: ask good questions
  • Intellectual autonomy: think for oneself
  • Intellectual humility: admit one’s limitations
• Keep learning process on track
  • Intellectual attentiveness: look and listen
  • Intellectual carefulness: avoid errors
  • Intellectual thoroughness: go deep to understand
• Deal with obstacles
  • Open-mindedness: think outside the box
  • Intellectual courage: take intellectual risks
  • Intellectual tenacity: embrace struggle

• Curiosity: Ask good questions | to learn | discerning when to start, continue, or stop the inquiry | mediating between indifference and fixation.
• Intellectual autonomy: Think for oneself | to achieve independent thought or self-assuredness | discerning when to yield to others or differentiate from others | mediating between conformity and radicalism.
• Intellectual humility: Admit one’s limitations | to recognize what one is able or unable to do or to locate oneself in the context of others’ interests | discerning when to assert oneself or to stand back | mediating between arrogance and self-deprecation.

• Intellectual attentiveness: Look and listen | to remain alert to details | discerning when to tune out or to focus more intently | mediating between distractedness and preoccupation.
• Intellectual carefulness: Avoid errors | to assure or control the quality of one’s output | discerning when to ease up or to double down on quality control | mediating between sloppiness and perfectionism.
• Intellectual thoroughness: Go deep to understand | to ensure sufficiently complete coverage or treatment | discerning when to fill gaps or let well enough alone | mediating between superficiality and meticulousness.

• Open-mindedness: Think outside the box | to consider new or unfamiliar ideas | discern which ideas to dismiss or to entertain an idea | mediating between narrow-mindedness and gullibility.
• Intellectual courage: Take intellectual risks | to allow for bold action despite potential for failure | discerning when to tolerate more or less potential for failure | mediating between cowardice and foolhardiness.
• Intellectual tenacity: Carry on | to continue toward learning objective, even when challenged | discerning when to persist and when to stop trying | mediating between fickleness and stubbornness.

• Intellectual virtues are developed by practicing them
  • Courage: take risks and learn from the consequences
  • Humility: own your limitations rather than shy away from them
• Intellectual virtues are developed through guidance
  • See them modeled, by mentors and by peers
  • Receive guidance
• Connect love of knowledge to self-learning, problem-solving

• Value innovation, respect the fundamentals
• Practice pragmatic, principled pluralism
• Apply wisely and well all methods that can help achieve technical excellence to learn from, about, and with data
  • Classical, conventional, and innovative
  • Statistics and machine learning
  • Correlational, causal, and predictive inference
  • Analysis, synthesis, and forecasting
• Principled pluralism allows honest disagreement about methods, results, and interpretation.

• Foster public trust, motivated by public service to conduct itself ethically
• Protect privacy
• Ensure that data and methods are radically open and transparent

• Basic or intermediate data skills
• Come from any discipline to do good things with data
• Everyone who wants to do good things with data has the intellectual support to do so, accepting that they must proceed with rigor and stand behind their work
• Practitioners learn continuously and show how modern tools and methods solve modern problems

• Go deep on data science methods
• Help ensure that everyone who wants to do good things with data, can
• Set norms for the practice of data science and for learning from, about, and with data
• Enable, guide, correct, and empower practitioners to proceed with rigor and stand behind their work

• Ensure that practitioners and experts have resources and direction to achieve good things with data
• Foster and reward curiosity
• Invest in learning (not just training)
• Encourage innovation and interesting mistakes
• Advocate for means to enable practitioners and experts to continue increasing capability, efficiency, and effectiveness
• Hold practitioners and experts to account for producing knowledge learned from, about, and with data

• Work in community with practitioners, experts, and managers
• Literate in the value of data to help learn things about the world
• Know how to assess, use, and advocate for learning from data
• Help ensure supportive resources to enable learning

• Guide technical skills
• Model intellectual virtues
• Exercise practical wisdom
• Create opportunities to practice intellectual virtues

• Fellowships (EIS, PE, PHI, PM) and some structured programs (DSU) foster communities of peers
• Students and ORISE tend to lack organized community
• Interest groups and workgroups
  • SAG, RUG, PUG, HERG; SciComp
  • CDC Data Hub communities of practice (e.g., CMS, MarketScan)

• Engage data practitioners and experts as co-leaders
• (put the data in DMI)
• Engage early-career professionals as co-leaders
• (put the modernization in DMI)
• Emphasize that science leads technology and that technology opens possibility but follows science
• Espouse scientific quality and rigor as primary goods
• Promote leadership, empowerment, and wise innovation as the affirming manifestation of governance
• Find out from prospective talent what they want and need in order to be recruited and retained

• A progressive culture
  • remains rooted in history
  • continues to learn from old data in new ways
  • anticipates the future
  • handles evolving demands to keep up with fast-moving methods, technology, and tools

• Data science: a set of core activities to ask good questions and line up the tools to answer them rigorously using data
  • Practice of data science crosses disciplines: ask and answer
  • Virtues drive commitment to public service, self-learning (to keep up!), and autonomy to engage and solve problems
• Elements to foster a data science culture
  • Know-how to bridge domain knowledge and methods
  • Love of knowledge to drive problem-solving with data
  • Community provides direction, wisdom, and support

• Public health sector faces constant challenges to stretch modest resources, to anticipate and respond to threats, and to promote population health
• Cultivating a progressive data culture in the present will best position the field of public health as ever-ready to learn from and act on data

• Traditional data architectures can’t efficiently handle
  • • Volume — the size of the dataset
    • Variety — data from multiple repositories, domains, or types
    • Velocity — rate of flow
    • Variability — the change in other characteristics
• Only the above 4 Vs drive the shift to parallel architectures for data-intensive applications, in order to achieve cost-effective performance.

• Practitioners with sufficient knowledge in the areas of business needs, domain knowledge, analytical skills, and software and systems engineering to manage the end-to-end data processes in the data life cycle
• Overlapping skills including data analysis, analytical applications, big data engineering, algorithms, domain expertise, statistics and machine learning
• Use expertise in one [corrected: or] more of these domains to solve complex data problems

Interdisciplinary field of inquiry in which quantitative and analytical approaches, processes, and systems are developed and used to extract knowledge and insights from increasingly large and/or complex sets of data.

• Multidisciplinary and interdisciplinary approaches to extracting knowledge or insights from data
  • Relies on processes and systems (mathematical, computational, and social) to derive information or insights from data
• Synthesize most relevant parts of the foundational disciplines to solve particular classes of problems or applications
• Create novel techniques to fill between those disciplines where no approaches yet exist