The project was approved by the Colorado Multiple Institutional Review Board and the University of New Mexico Human Research Protections Office and is registered on ClinicalTrials.gov (NCT02515578, protocol identifier 15–0403). ENSW is a cluster randomized trial with 2 active interventions and an external comparison group that received no intervention. As we compared 2 active intervention arms and there were ongoing national initiatives focused on cardiovascular care (eg, Million Hearts), the external comparison group was included to control for temporal trend. Because of the intent to implement community engagement interventions at the community and regional level and the timeline of the study, which required knowing which regions were in which study arm before completing practice recruitment, we used geographic-based covariate constrained randomization to allocate geographic-based regions (26 in Colorado, 16 in New Mexico) to intervention arms to achieve balanced study arms with respect to underlying populations and resources. This approach is described in detail elsewhere. This study was powered to detect a 0.47 effect size difference between study arms on practice-level outcomes with no clustering (ie, single measure per practice) or a medium linear trend effect (differences increase from 0 to .5 SD over time). An external cohort of similar practices was obtained from the DARTNet Institute, a nonprofit institute that coordinates and supports research, quality improvement, and safety activities across multiple research networks through the collection of electronic health data.[38,43]
As detailed below and summarized in Table 1, we compared the effectiveness of a standard practice transformation support intervention to practice transformation support enhanced by patient and community engagement. Twenty "practice transformation organizations" (PTOs) across Colorado and New Mexico provided the practice facilitator and clinical HIT advisor (CHITA) services for the practices. Standardized training was provided by the research team to experienced practice facilitators and clinical HIT advisors from the participating PTOs in a group setting. In some cases, the same individual served in both practice facilitator and CHITA roles, depending on individual skills and PTO resources. Required field notes tracked contacts and content of all facilitator and CHITA activities and were reviewed regularly by the research team to ensure fidelity to the intervention components.
All Practices, Standard and Enhanced, Received the Following.
Practice assessment with feedback to practice. A baseline practice survey assessed practice culture and change capacity, recent or ongoing practice transformation efforts, and current level of implementation of Patient-Centered Medical Home (PCMH) components and cardiovascular care. Assessments were aligned across EvidenceNOW cooperatives. Results of assessments were summarized in feedback reports to practices to help initiate the quality improvement process through a reflective discussion of practices' needs and priorities.
Practice facilitation with quality improvement team meetings. Active practice facilitation (and the intervention baseline) began with the feedback report to the practice. The active facilitation phase focused on monthly meetings of a practice improvement team, consisting of diverse representatives of the practice (eg, front office, medical assistants, and clinicians). The facilitator helped the team select quality improvement activities to improve care for patients with cardiovascular risk and then assisted the team in using data and quality improvement methods for rapid cycle change. The goals were to implement practice transformation activities based on the first six Building Blocks and evidence-based cardiovascular risk interventions. Practice facilitation lasted for 9 months.
Collaborative learning sessions. Representatives from each practice participated in 2 collaborative learning sessions during the course of their intervention, providing an opportunity to share lessons learned with other practices working on the same interventions.
HIT support. On-the-ground HIT assistance by CHITAs aimed at developing data capacity for quality measures and population management and linkage to our quality measure data aggregation system. CHITAs met with practices approximately monthly during the 9-month intervention. CHITAs supported quarterly clinical quality measure (CQM) reporting and provided measure feedback reports to the practices.
Cardiovascular toolkit. An online hub of support materials and e-learning modules aligning with the Million Hearts Campaign and the first six Building Blocks were offered to practices, focusing on the following areas: implementing evidence-based guidelines for aspirin therapy and treatment of hypertension (HTN) and cholesterol; promoting systems to support self-management, such as telephone follow-up and home blood pressure monitoring; linking primary care practices and community and public health resources; implementing multidisciplinary team-based care; and implementing a comprehensive approach to smoking cessation, including referrals to community resources.
Enhanced Intervention Practices Received 2 Additional Elements.
Regionally-tailored patient engagement materials. Boot camp translation was used to produce patient engagement materials based on practice and patient input regarding the local context. Within regions randomized to the patient engagement arm, we convened 4 regional groups of 16 or fewer individuals representing patients, clinicians, staff members, and local public health. This process and examples of the locally developed materials are described elsewhere. The patient engagement materials developed through this process were offered only to the enhanced arm practices.
Patient and family advisory councils. ENSW provided centralized resources and trained the practice facilitators to support the formation and active use of patient and family advisory councils. Notably, e-learning modules were developed to assist practices in engaging patients in the practice transformation process.
Implementation. The conceptual and operational framework for the practice facilitation intervention was based on the first 6 Building Blocks for High-Performing Primary Care. This was assessed using an Implementation Tracker in which practice facilitators rated practices' progress on the 6 building blocks (no activity, in progress, or complete) at baseline and 3, 6, and 9 months (active intervention period). Building Blocks were scored as sum of items (reversed, if necessary), scaled from 0 to 100. Cronbach's α for items within each subscale ranged from 0.79 to 0.92.
Primary effectiveness outcomes (ABCS measures) were obtained from practice-level CQMs on cardiovascular risk management (aspirin use, blood pressure, cholesterol, and smoking) collected quarterly. Each measure was based on numerators and denominators from the previous 12 months. Quality assurance strategies were imposed for all measures, and any data points that deviated from expected values were verified and corrected, if necessary, by the data manager and the CHITA. Measures from the DARTNet external comparison practices were used as reported to the research team after applying the quality restrictions used for ENSW practices. The ABCS measures are described below, as follows:
Aspirin use. Percent of patients 18 years and older with ischemic vascular disease (IVD) with documented use of aspirin or other antithrombotic.
Blood pressure management. Percent of patients aged 18–85 who had a diagnosis of HTN and whose blood pressure was adequately controlled (<140/90) during the measurement year.
Cholesterol management. Percent of patients from three categories considered at high risk for CVD who were prescribed or were on statin therapy.
Smoking cessation support. Percent of patients aged 18 years or older screened for tobacco use one or more times within 24 months and who received cessation counseling intervention if identified as a tobacco user.
A team or an individual designated by the practice completed a practice survey at baseline that included key practice demographic information.
Descriptive statistics were computed for baseline practice characteristics, initially testing for differences between: (1) different intervention arms and (2) practice dropouts versus nondropouts. For analyses of differences between study arms, practice-level covariates were screened in bivariate analyses and initially included in multivariate analysis if they were related to the outcome at P < .1, differed between treatment arms, or were associated with dropout. When there was high collinearity among covariates, a single covariate was chosen to represent that domain. In general, we used intent-to-treat analyses that used all available data, assuming ignorable missingness.[49,50] Because normality assumptions were not met for the CQMs, we used generalized linear mixed models (mixed effects beta regression) with random intercepts for practice to analyze the data. This approach has been shown to be more suitable than linear mixed models for continuous data that is bounded (0, 1) and often skewed, such as proportions.[51–53] To include all valid observations, a small constant (0.001) was added or subtracted from observations that had a value of 0 or 1. Building Block outcomes were analyzed using general linear mixed models with a random effect for practice. We assessed overall improvement over time (main effect for time) as well as differential improvement by study arm or ENSW vs external comparison group (time by group interaction term) for all analyses. In addition, we examined improvement in Building Blocks of High-Performing Primary Care (secondary/intermediate outcomes) as potential mediators of improvement in ABCS quality measures (see conceptual model) using general linear mixed models.[54,55] Hypothesis tests were 2-sided with α = 0.05 or p values reported. All statistical analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC).
External Comparison Group. A total of 522 practices in the database provided by DARTNet Institute had complete data on practice ownership (FQHC, private, and hospital) and location (rural/urban), along with county-level census data on population demographics (% over age 65, % Hispanic, % black, % female, and median household income). From the subset of 457 practices that had nonzero proportions on at least 3 of the 4 CQMs during the baseline 12-month performance period ending December 31, 2016, we selected a random sample of 200 practices that was comparable to ENSW practices. Adapting our covariate constrained randomization procedure, we (1) generated 1,000 random samples of 200 practices from the 457 eligible practices in the DARTNet database, (2) calculated a balance criterion for external comparison group versus ENSW (sum of the absolute values of the standardized distances between means of each random sample and the overall mean from ENSW practices on the above variables), (3) identified a candidate set of random samples as the top 30% overall that had at least 14.5% rural (a total of 50 random samples of 200 practices), and 4) randomly selected a single random sample of 200 practices from the candidate set as the external comparison group. Data for the comparison group were extracted and processed in mid-2017. Analyses comparing active intervention practices (all ENSW practices) to the external comparison group used similar approaches to those described above.
J Am Board Fam Med. 2020;33(5):675-686. © 2020 American Board of Family Medicine