Data Validation Process

Validating your data is important to ensure that each calculated measure rate is an accurate, reliable reflection of the care that has been delivered and the outcomes a patient has experienced. Community Health Centers (CHCs) should have a validation process in place to ensure the accuracy of PHMI/HEDIS measurement.

Data systems, including the electronic health record, population health management system and other analytics tools that generate PHMI/HEDIS measures and reports should meet the highest levels of data quality so that management and employees may rely on data for care coordination, decision making, and ongoing improvement. Each organization should develop a data validation process to ensure reported measures are systematically assessed for accuracy, completeness, and timeliness to ensure usability of the measures.

Ensuring accurate and reliable data in a health center requires participation at all levels of the organization. The following table describes functional roles and activities that are important for establishing a systematic and sustainable approach to data validation:

All reports that are generated should have a process for conducting consistency and reasonability checks to identify issues with data accuracy, completeness, or timeliness. Data validation should be conducted by staff trained on the measures they are validating, criteria for compliance, and potential data sources.

Verification Checks for Internal Reports

All reports generated internally should include the following data verification checks for accuracy and completeness. This should occur prior to publishing for staff or submitting externally to ensure that:

• The measure computation conforms with the measure’s numerator and denominator definitions and exclusion criteria (UDS, HEDIS, other)
• The appropriate EHR/PHM fields are being used for the measure
• The numerator is not greater than the denominator (the numerator can be equal to the denominator but not greater than it).
• The denominators appear reasonable and in alignment with other sources and knowledge.
• The right population is included in the report and sub-populations are not greater than parent populations.
• Segmented populations (e.g., race/ethnicity) align with expectations and are reasonable given the segmented population breakdown.
• The data entered is accurate (e.g., A1c value of 6.5 vs 65).
• All structured external interfaces (e.g., lab, pharmacy) are functioning properly.

 

Reconciliation

Data validation standards should include reconciliation routines:

• Identify unexplained variation of 10% or higher over time and/or among providers/teams/site and investigate inaccurate data noted by those reviewing reports when sharing in periodic management/clinical/ops forums.
• Compare data against past reports and other reports (UDS, QIP eReports, HEDIS lists received directly from the payer, attribution lists segmented by age, gender) to verify alignment.
• Compare against external norms and benchmarks (e.g., CDC, UDS, and NCQA) to check relative value of data and measures. For disease-based reports, prevalence percentages match national and regional averages.
• Investigate possible data inaccuracies through additional reports/searches and by reviewing data with sites/providers/staff where variation exists.
• Assess inter-rater reliability when unsubstantiated discrepancies of greater than 10% are found in data verification and validation and conduct Primary Source Verification (PSV) when there are conclusive discrepancies or as required by reporting standards (e.g., HEDIS measures).

 

Passing Data Validation

A robust data validation process will allow CHCs to identify and remediate issues with data quality and reliability and ensure measure rates are an accurate reflection of patient care received and services provided. A minimum threshold for “passing” data validation standards is an impact of less than 5% percent on the rate. Impacts can be estimated based on the results of the data checks and primary source verification described below.

If data validation results indicate a known deviance greater than 5%, CHCs should:

1. Indicate data validation issues on the reporting tool submission.
2. Develop and implement a remediation plan.

 

Automation

As a best practice, CHCs should migrate from a manual validation process to a mostly automated process. Adopting a more automated approach offers a faster, more efficient and consistent way to extract the data for the core HEDIS measures for PHMI. It also offers improved data quality and integrity in other business processes and programs, like CalAIM or MCP P4P.

 

Data Validation Examples

Below are two real-life scenarios in which a possible data validation issue is identified and reconciled.

Example 1: Not all eligible patients are being included in the measure

The CDC reports that about 12% of the population has diabetes yet this can vary considerably by demographic, particularly race and ethnicity. If, while assessing the magnitude of diabetes measure denominators, a CHC finds significantly more or less diabetes patients than expected for the patient population served, the report should further evaluated. This may include review of report criteria to ensure it matches measure specifications and may also include assessing how diabetic patients are documented in the EHR.

Example 2: Measure rates changed unexpectedly

A cervical cancer screening report showed that rates had been declining over the past few months. When this was discussed in a quality improvement meeting, care team members did not understand why this was happening and, in fact, felt that it should be improving based on outreach efforts. The data analyst confirmed that the report was configured appropriately according to measure specifications. Health IT staff were then asked to investigate and they uncovered an error in a laboratory interface following a software update. After refreshing the data, the report showed that, in fact, cervical cancer screening rates were improving.

Primary Source Verification (PSV) is an approach that entails reviewing the original source documents or systems where the data was first created. PSV should be conducted when there are discrepancies in data or measures against other data/information sources or there are notable shifts between reporting periods, as described above. PSV should also be conducted as required by MCOs and other external reporting platforms.

• The threshold for notable discrepancy is typically 10%, but this may be higher when populations/denominators are low.
  • Begin with a random sample of 5 patients for each measure. If the primary source verified the information contained in the patient-level file for those patients, the measure passes PSV.
  • If discrepancies are detected, select a larger sample (up to an additional 45 records, as a best practice), then assess results to determine if deficiencies are detected. If none, PSV is complete. If yes, identify gaps and determine whether the gaps are isolated or pervasive. Develop an appropriate remediation strategy and report on progress through the data governance function.
• Conduct PSV as an initial process with new measures and reports and continue with each subsequent reporting run until no issues are found. Also conduct PSV any time there is a material change in how the measures are pulled or data sources used.

An Example of Primary Source Verification

A patient who was identified as numerator-compliant for the colorectal cancer screening measure should have evidence in the medical record of factors that comply with the measure. This would include:

• Patient is 45 to 75 years of age.
• Patient has documented evidence of a colorectal cancer screening within the time frame allowed for the particular type of colon cancer screening received.

Recommended Approach to Primary Source Verification

PSV can be a resource-intensive process. For PHMI, initial PSV could consider a random sample of five patients from each measure. If the primary source verified the information contained in the patient-level file for those patients, the measure would pass PSV. If discrepancies were detected, the CHC should:

• Select a larger sample (up to an additional 45 records, as a best practice).
• Assess results of the larger sample to determine if deficiencies are detected.
  • If no, PSV is complete.
  • If yes, the CHC should identify gaps and determine whether the gaps are isolated or pervasive.

Based on findings, the CHC should develop an appropriate remediation strategy and report data validation issues on its data reporting tool (DRT) submission. CHCs should use PSV to validate their PHMI/HEDIS rates:

• As an initial process and continuing with each submission until no issues are found during PSV.
• Any time there is a material change in how the measures are pulled or data sources used.

Use Figure 7.2: Validation Checklist 2: Primary Source Verification below as a checklist when conducting primary source verification of the overall population in the database or master file from which the measures are calculated, as well as rates for each core HEDIS measure for PHMI.

FIGURE 7.2: VALIDATION CHECKLIST 2: PRIMARY SOURCE VERIFICATION