Children - Appendix C


C: Developing a Robust Measurement Strategy



The visual below illustrates the key milestones in the development of a robust measurement strategy.

Phmi Measurementstrategymilestones



Figure 24 provides guidance on each of these milestones as you work to put in place a robust yet practical measurement strategy.





The overall goal(s) of the improvement effort. “What are we trying to accomplish?”

We often recommend sub-aims to focus your team on intermediate goals. You can develop data-informed specific, measurable, achievable, relevant, time-bound, inclusive and equitable (SMARTIE) goals focused on increasing adherence to specific AAP preventive care guidelines year over year among attributed patients or subpopulations of patients.

Our overall aim is to increase adherence to AAP preventive care guidelines among patients.

Example sub-aims: By December 2025, 95% of children 18 months of age will receive all recommended measles-mumps-rubella (MMR) vaccines or have documented parental declination.

By December 2025, we will increase the percent of children zero to 30 months of age whose parents have Spanish as a preferred language who were sent a reminder in Spanish for recommended immunizations during the measurement year.


A general abstract notion (approach, thought, belief or perception) related to the aim(s) of focus.


Specific objective ways to determine the extent to which an aim has been met or to determine if there has been improvement in the concepts of focus. Measures help us to answer the second question in the model for improvement: “How will we know that a change is an improvement?” Measures generally fall into one of three types:

  • Outcome measures: Measure the performance of the system(s) of focus and always relate directly to the aim(s). Outcome measures focus on the end results and offer evidence that changes are actually having an impact at the systems level.
  • Process measures: Pertain to the activities, steps or actions taken within the system(s) of focus that are believed to be most strongly related to improving the outcome(s) of focus. These measures help evaluate efficiency, effectiveness and consistency. Process measures are essential for understanding how well the system is working and can be early indicators of improvement.
  • Balancing measures: Look at a system from different directions and evaluate dimensions, such as the effects a change may have on other parts of the system. These also include ways of assessing unintended consequences further upstream or downstream.

See below for example outcome, process and balancing measures.

For the following examples, we will examine the percentage of two-year-old children who have received the 10 HEDIS-reported vaccinations.

Operational definitions

Detailed descriptions in quantifiable terms of what to measure and the steps to follow to do so consistently each time and over time. The operational definitions help make the measures clear and unambiguous and often contain criteria for inclusion and exclusion and numerator/denominator.

Percentage of two-year-old children who have received the 10 HEDIS-reported vaccines, operational definition. See CMS117v11 for more information.

Stratifying these data by type of vaccine (e.g., Hib, MMR) can provide additional information and insight to inform improvement efforts.

Data collection plan

A detailed set of instructions that generally includes:

  • Who (specifically) will collect the data.
  • How (specifically) the data will be collected.
  • Where and how the data will be stored.
  • When the data will be collected.
  • How often (e.g., frequency) the data will be collected.

Percentage of two-year-old children who have received the 10 HEDIS-reported vaccines, data collection plan.


  • Nursing staff or other medical personnel perform immunizations and input it into the EHR. Pre-visit planning to align immunizations with regular well-visits.
  • Receptionist scheduling to align with well-visit times.


  • Immunizations should be entered in a discoverable format within the EHR.
  • Reports should be run on a monthly basis within the electronic health record to collect data.

Where and how the data will be stored:

  • Data should be stored within the patient’s health record.
  • External data sources, such as Health Information Exchanges (HIEs) such as Californian Immunization Registry (CAIR2) should be reconciled with the health record.

When the data will be collected:

  • Data will be collected during patient visits on an ongoing basis.

How often:

  • Evaluation and gap reports should be run on a monthly basis to evaluate current status and determine current patients who are behind on their immunizations.
  • Stratifying these data by provider can also help identify opportunities for improvement.

Analysis and action

The process of analyzing the data, including instructions for the analysis and visualization of the data, disseminating the data to relevant parties, and using the data to track progress and guide improvement efforts.

The improvement team reviews the percentage of two-year-old children who have received the 10 HEDIS-reported vaccines on a monthly basis, and the care team and panel manager review the data subsequently once per quarter or every six months for ongoing monitoring.