Methods
Study Setting
We used data collected from medical groups participating in a statewide performance measurement initiative in California during 2007. These medical groups, a mix of free-standing physician practices and independent practice associations, provide healthcare for ~90% of the state's insured population, and range in size from 20 physicians to several thousand physicians.
Patient Survey Eligibility
We combined data collected from two sampling frames in 2007, a medical group level survey and an individual physician level survey (Fig. 1). The medical group level survey sample identified all patients aged 18 years and older who had a visit with either a primary care or a specialist physician during the 10-month period from January to October 2007. A random sample of 900 patients per medical group was selected and linked to a single physician (primary care or specialist) based on their visit history. The survey was administered in a three-stage process, with an initial mailing, a follow-up mailing and a final attempted telephone interview. The overall response rate was 37%.
(Enlarge Image)
Figure 1.
Enrollment of patients being treated for chronic conditions. Patients with chronic disease were identified using surveys and were included in the study based on receiving care from a primary care physician or specialist within one of the medical groups self-reporting clinical performance data.
The individual physician level survey sample was derived from a subset of the medical groups who volunteered. Patients were identified based on the presence of an office visit with an individual physician during the 10-month period from January to October 2007. A random sample of 100 patients per physician was then selected. These patients were exclusive of the 900 patients sampled from that site for the medical group level survey. The survey was administered in a two-stage process, with an initial mailing followed by a second mailing to non-responders. The overall response rate was 34%.
Survey sample sizes were chosen with the goal of achieving a minimal threshold reliability for provider-level comparisons of 0.70. The response rates achieved are consistent with those achieved in other statewide surveys, where non-response analyses have not demonstrated any biases or challenges to the integrity of provider-level analyses of patient experience measures.
Survey Instrument
Patient experiences of care were measured based on the CAHPS® Clinician and Group Survey endorsed by the National Quality Forum. Self-management support was assessed using a validated composite measure based on five individual survey items (Table 1).
We identified patients receiving care for a chronic disease by asking the survey question 'In the last 12 months, did you have any health problems or conditions for which you took medicine or got care for 3 months or longer?'. Patients also identified specific chronic conditions from a pre-specified list of diseases, and were allowed to select the presence of more than one chronic condition.
Clinical Performance Measures
As part of the statewide performance measurement program, clinical data were voluntarily submitted by 89 medical groups using a combination of administrative and medical record data. These 89 medical groups represent 51% of all groups statewide (Fig. 1). The remaining groups were not able to submit data due to the lack of information systems infrastructure to facilitate large-scale data extraction on clinical quality measures. The submitted data are processed by the National Committee for Quality Assurance and performance measures are calculated using standard specifications from the Health Plan Employer Data and Information Set (HEDIS). We evaluated measures for the management of asthma, cardiovascular disease and diabetes (Table 2).
Composite Measure Creation
We created four composite measures of patient experience, one composite measure of disease self-management support and two composite measures of clinical quality. The patient experience composites have been validated in prior studies and included one measure of the quality of clinical interactions, three measures of organizational features of care (integration of care, office staff and organizational access) and one measure of self-management support (Table 1). Composite measures of clinical quality were developed for diabetes, cardiovascular disease and asthma management by separating process of care measures from clinical outcomes (Table 2). All composite measures demonstrated good internal validity, with Cronbach's α scores exceeding a minimum threshold of 0.70 with the exception of the integration of care, which demonstrated a score of 0.63 (Table 1 and Table 2).
Numeric composite scores for clinical quality measures and patient experience measures were calculated using the adjusted half-scale rule to produce ratings on a scale from 0 to 100, with higher scores representing either better patient experiences or superior clinical quality. This involves first transforming the individual item to a 0–100 scale. HEDIS scores are already ranked as such, while patient survey responses require conversion (e.g. a five-point Likert response item would generate a score of 0, 25, 50, 75 or 100).
Correlation Analyses
We calculated non-parametric Spearman's correlation coefficients between individual process of care measures and their relevant outcome measure at the medical group level, including those between: (i) annual HbA1c monitoring and achieving HbA1c control (<9%) for patients with diabetes, (ii) annual low-density lipoprotein (LDL) cholesterol monitoring and achieving LDL cholesterol control (<130 mg/dl) for patient with diabetes, and (iii) annual LDL cholesterol monitoring and achieving LDL cholesterol control (<130 mg/dl) for patients with cardiovascular disease.
We next calculated correlation coefficients at the medical group level between the two composite measures of clinical quality (process composite and outcome composite) and the five composite measures of patient experiences, for a total of 10 correlations. While the clinical composite scores were calculated based on patients with specific chronic diseases, the correlations with patient experiences were performed among all patients screening in for chronic disease, regardless of the specific condition.
For all correlation coefficients, we calculated the convergence estimates, or adjusted correlation coefficients, to represent the true correlation that would be observed between the two composite scores in the absence of measurement error associated with creating the composites. The convergence estimates were calculated using the Spearman–Brown reliability coefficients for the patient experience composite (α1) and the clinical quality composite (α2), along with the Spearman correlation coefficient (ρ) in the following manner: convergence =
.
This study protocol was approved by the Human Studies Committee at Tufts Medical School. All analyses were carried out using the STATA statistical package, version 9.0.