Detecting Disengagement From HIV Care Before It Is Too Late

Development and Preliminary Validation of a Novel Index of Engagement in HIV Care

Mallory O. Johnson, PhD; Torsten B. Neilands, PhD; Kimberly A. Koester, PhD; Troy Wood, MA; John A. Sauceda, PhD; Samantha E. Dilworth, MS; Michael J. Mugavero, MD; Heidi M. Crane, MD; Rob J. Fredericksen, PhD; Kenneth H. Mayer, MD; William C. Mathews, MD; Richard D. Moore, MD; Sonia Napravnik, PhD; Katerina A. Christopoulos, MD, MPH


J Acquir Immune Defic Syndr. 2019;81(2):145-152. 

In This Article


Sample and HIV Index Item Characteristics

More than half (51.6%) of the sample of 3296 patients were older than 50 years, four-fifths were males (79.6%), 40.7% Black/African American, 44.7% White, and 67.0% identified as sexual minority (ie, gay, lesbian, bisexual, and other). Most participants were on ART (94.3%) and virologically suppressed (89.7%).

The average number of individuals who did not respond to any single item was 3.4%, with a median of 2.5% (Table 2). One item, "How often do you refill HIV medications on time?", permitted a "not applicable" response, resulting in 15.6% having not applicable or missing responses, because of 5.7% not being on ART and others likely because of those respondents obtaining automatic prescription refills. In general, the sample consisted of very well-engaged patients, with the majority of respondents endorsing the top 2 levels of engagement for all questions (Table 2).

Item Screening Step

The Hull method indicated that a single factor best represented the shared variance among the 13 HIV Index items. The scale ECV (0.904) and MIREAL (0.210) exceeded the recommended thresholds for unidimensionality. The item-level I-ECV and I-REAL indices of unidimensionality exceeded the recommended thresholds for all items, except for: "How important is it for you to set goals for your health?" (I-ECV = 0.734; I-REAL = 0.387), "How often do you refill your HIV medications on time?" (I-ECV = 0.719; I-REAL = 0.233), and "How important is it for you to stay informed about new HIV research findings?" (I-ECV = 0.573; I-REAL = 0.518). These 3 items were removed, yielding a reduced 10-item scale with one latent factor (Table 3 for the retained items).

Exploratory Factor Analyses

EFA of the 10 retained items indicated excellent fit for the one-factor solution: χ 2(35) = 273.06, P < 0.001; RMSEA = 0.064; CFI = 0.990; and SRMR = 0.032. Factor loadings indicated strong factor–variable relationships with most items' loadings exceeding 0.70 (Table 3). Refitting the one-factor EFA using MI data sets yielded highly similar global model fit results (mean χ 2(35) = 288.90, SD = 9.79; RMSEA = 0.066; CFI = 0.989; and SRMR = 0.022), factor loadings, and 95% confidence intervals (CIs), suggesting robustness of these results under different missing data mechanisms (Table 3).

Confirmatory Factor Analyses

CFA using the validation subsample yielded highly similar values of global model fit statistics to those found in the factor extraction subsample and indicated strong support for the one-factor structure of the HIV Index: χ 2(35) = 282.06, P < 0.001; RMSEA = 0.065; CFI = 0.988; and SRMR = 0.031. Refitting the CFA using MI data sets yielded highly similar global model fit results (mean χ 2(35) = 294.97, SD = 9.79; RMSEA = 0.067; CFI = 0.988; and SRMR = 0.020). As shown in Table 3, the pattern of factor loadings was also highly consistent with that found in the EFA for both the original and imputed data sets. Taken collectively, our factor analysis results indicate robust support for a single engagement in HIV care latent factor that represents the shared variance among the 10 retained items.

Reliability Analyses

Cronbach's alpha for the 10-item scale in the factor extraction subsample was 0.886. The corresponding alpha value in the validation subsample was 0.878. Alpha did not increase appreciably if any items were removed from the reliability analysis in either subsample. These results indicate the proposed 10-item HIV Index has strong and consistent reliability.

Predictive Validity Assessment

Logistic regression analyses revealed negative associations between the HIV Index scale score and detectable viremia (OR = 0.66; 95% CI: = 0.60 to 0.74), such that the odds of detectable viremia were 34% lower for each SD increase in the Index score. In addition, the HIV Index score was negatively associated with missing 2 or more HIV-related medical appointments in the past year (OR = 0.75; 95% CI: 0.69 to 0.83). The HIV Index score was positively associated with attending one or more HIV care visits in each of the 2 six-month windows 180 days before the Index measurement was taken (OR = 1.11; 95% CI: 1.02 to 1.21). The HIV Index score was also positively correlated with the proportion of HIV care appointments kept in the year before Index measurement (r = 0.13, P < 0.0001). In addition, positive associations were also observed with increases in the HIV Index scale score and being on ART (OR = 1.43; 95% CI: 1.27 to 1.62), 100% VAS adherence (OR = 1.37; 95% CI: 1.25 to 1.51), and self-rated ART medication adherence (OR = 1.72; 95% CI: 1.60 to 1.85; proportional odds test χ 2(4) = 3.01, P = 0.55).

Convergent and Discriminant Validity Assessment

Table 4 displays the sample and imputation-based Spearman correlations. The HIV Index score, computed as a single score representing the sum of the 10 individual items, was significantly negatively associated with depressive symptoms, stimulant use, problem drinking, HIV-related stigma, and anxiety. The HIV Index score was not associated with age, Black race, gender minority status, or the time that the participant was in the CNICS cohort. Sample-based correlations and MI-based correlations differed little, suggesting robustness of the correlation results under different missing data mechanism assumptions.