1 Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD.
2 Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD.
3 Center for Urban Epidemiologic Studies, The New York Academy of Medicine, New York, NY.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
HIV; needle-exchange programs; program evaluation
Abbreviations: ACASI, audio computer-assisted self-interview; ALIVE, AIDS Link to Intravenous Experience; AOR, adjusted odds ratio; CI, confidence interval; ELISA, enzyme-linked immunosorbent assay; HCV, hepatitis C virus; HIV, human immunodeficiency virus; IDU, injection drug user; NEP, needle exchange program; OR, odds ratio
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The most desirable measure of the effectiveness of NEPs is a reduction in the incidence of blood-borne infections. Most studies have shown that NEPs can be effective in reducing drug-related risks associated with human immuno-deficiency virus (HIV), hepatitis C virus (HCV), and hepatitis B virus infection (48
). However, some studies have shown higher rates of HIV seroconversion among NEP attendees relative to nonattendees (9
11
), and others have suggested no protective effect of NEP on HCV seroconversion (12
). Some authors have attributed these findings to selection bias, since NEPs tend to attract IDUs who engage in higher risk injection activities than do nonattendees (11
, 13
15
). Hahn et al. (13
) showed that HIV incidence rates were higher among IDUs who later began attending an NEP in San Francisco, California, compared with those who did not, which supports the existence of a strong selective effect. Similarly, describing possible confounders in studies evaluating efficacy of NEPs, Hagan et al. (15
) found that IDUs who participated in NEPs had a pattern of drug use that placed them at higher risk of acquiring blood-borne infections. These phenomena complicate evaluations of NEP through observational study designs, since comparisons of attendees and nonattendees of NEP tend to be biased (1
).
The majority of studies evaluating NEP effectiveness rely on self-reported behaviors. It has generally been shown that the self-reports of most drug users are valid (16). However, recent studies comparing audio computer-assisted self-interview (ACASI) with interviewer-administered surveys indicate that IDUs tend to underreport sensitive behaviors such as needle sharing (17
, 18
) and overreport protective behaviors such as condom use and syringe disinfection (19
). We hypothesized that potential differential misclassification due to discrepant self-reported use of NEPs could under-estimate a protective association of NEP on HIV seroconversion. For example, if NEP attendees at higher risk for seroconversion overreport NEP attendance, the impact of NEP attendance on HIV seroincidence could be underestimated. The purpose of this study was to investigate whether drug users at high risk of HIV seroconversion were more likely to overreport attending NEPs. Further, we determined the extent to which a potentially protective association of NEP on HIV seroconversion may be underestimated in studies that rely on self-reports of program attendance. This analysis has important implications for future studies evaluating NEP effectiveness.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Participants also answered detailed interviewer-administered questionnaires that included demographics, risk behaviors for HIV infection, health service utilization, and health status. After the introduction of the Baltimore City NEP in 1994, the questionnaire was amended to include questions about attendance at the NEP in the previous 6 months. In addition, participants were asked whether, during the previous 6 months, they had obtained NEP needles for someone else and whether another person had obtained NEP needles for them. At each visit, participants provided locator data to facilitate active and passive follow-up and to confirm their identity at subsequent study visits; these data included the last four digits of their Social Security number, gender, date of birth, and mother's maiden name. As described below, the use of this unique identifier enabled cross-validation of self-reported attendance at NEP sites. This study received approval from the Committee on Human Research at the Johns Hopkins Bloomberg School of Public Health.
Baltimore NEP
The Baltimore City Health Department implemented an NEP that was authorized by state legislation on August 12, 1994. Participants and staff were exempt from prosecution for possession of drug paraphernalia. NEP services were provided through a recreational vehicle that serviced several locations across the city. Details of this program have been described previously (21).
New registrants in the NEP were assigned a unique identifier and a participant identification number that was used to record all future exchanges. The unique identifier was identical to that described above for the ALIVE study. The participant identifier was printed on a laminated card that clients were asked to present on subsequent visits to the program. This information was chosen to enable program staff to confirm the identity of clients in the event that they lost or forgot their registration card. However, these instances were uncommon, since NEP clients were exempt from state injection paraphernalia laws if they could present their NEP registration card to police. At each visit to the NEP, health department staff recorded the unique identifier of each client, the date of the visit, and the number of needles exchanged.
Participants received two syringes at their first visit to the facility regardless of whether they turned in a used syringe; subsequently, exchange was on a one-for-one basis with no upper limit. With each exchange, sterile alcohol pads, clean cotton balls, cookers, condoms, and HIV prevention brochures were made available. Referrals to subsidized drug treatment slots were available upon request.
Validation of NEP attendance
To enable a comparison of self-reported versus actual attendance at the NEP, ALIVE study participants were nonnominally linked to NEP program data by using each subject's unique identifier during the period August 1994 to October 1997, which corresponded to the first 3 years of NEP operation. Access to actual NEP visits was available through the NEP exchange data files, and self-reported NEP use was available through the semiannual ALIVE questionnaire data.
To provide the most conservative estimate of concordance between self-reported NEP attendance and actual program attendance, participants' reports of attending NEPs were considered concordant if NEP registration data confirmed that they had attended the program in the 7 months prior to the self-report (i.e., allowing a 1-month "grace" period). At each ALIVE study visit, participants had been asked by an interviewer-administered questionnaire whether or not they had attended an NEP to obtain syringes for another person and whether another person had visited the NEP to obtain needles for them during the previous 6 months (henceforth referred to as secondary syringe exchange). Using this information, we were able to account for further potential misclassification. Hence, visits in which self-report of NEP use did not match the NEP program data were defined as concordant if there was a self-report of secondary exchange during the previous 7 months.
When the self-report of the NEP use did not match the actual NEP visit and there was no evidence of secondary exchange, the visit was defined as discordant. Discordant visits were further subdivided into "underreports" and "overreports." Underreports of NEP attendance were defined as cases in which participants reported not using the NEP in the previous 7 months, whereas the program data indicated that they had, indeed, done so. Conversely, overreports were defined as cases in which the participant reported having used the NEP in the previous 7 months, but program data did not confirm this report.
Statistical analysis
Percent concordance and discordance were determined by comparing self-reports of NEP attendance with NEP registration data. Demographic and behavioral characteristics were compared according to discordant and concordant reports of NEP attendance. Since the hypothesis of interest was that overreports of NEP attendance among high-risk injection drug users could bias estimates of NEP effectiveness, we examined predictors of overreporting NEP by using logistic regression methods. In these analyses, underreports of NEP attendance were excluded because they were found to represent a very small proportion of study visits (i.e., 97 of 4,377 study visits (2.2 percent)). Since the analysis included serial measures for the same person, methods for analyses of correlated data, specifically, generalized estimating equation for binary outcomes with logit link, were used to determine which factors were independently associated with overreporting NEP use. Robust methods for logistic regression were used to provide standard errors adjusted by multiple observations per person (22) using an exchangeable correlation structure. Variables that were significant at the 10 percent level in bivariate analyses were entered in multivariate models. The final multivariate model identifying independent predictors of overreporting NEP attendance included only variables that were significant at the 5 percent level. All potential two-way interactions were explored.
To assess potential bias of self-reported NEP attendance on risk estimates of HIV seroconversion, bivariate and multivariate Poisson regression models were used to relate the risk of HIV seroconversion to predictors by using a log link with offset term that was the log person-years of follow-up time. Self-reported and actual NEP attendance were considered in separate models to permit comparisons of the adjusted relative incidence attributed to NEP use. In these models, self-reported and actual NEP attendance were both treated as binary time-dependent covariates that were updated at each semiannual period.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
When self-reports of NEP use were compared with program data, 310 (23.6 percent) of the persons had ever overreported use of NEP compared with 997 (76.3 percent) of those who were consistently concordant. When secondary exchange was taken into account, 86.5 percent of the study visits were concordant in terms of self-reports of NEP use compared with actual use of the program. A total of 13.2 percent of the visits were discordant with respect to self-reported versus actual program use. Of the discordant visits, 11.0 percent were overreports of NEP attendance, whereas 2.2 percent were underreports (table 1).
|
|
|
|
To assess potential bias of self-reported NEP attendance on risk estimates of HIV seroconversion, we compared models that included self-reported versus actual NEP attendance in a Poisson model in which the outcome was HIV seroconversion. The rate ratio for self-reported NEP use in the 6 months prior to the interview date was 0.98 (95 percent CI: 0.60, 1.61), whereas the odds ratio based on actual NEP attendance was 0.80 (95 percent CI: 0.46, 1.40). Results were similar after adjustment for known confounders, such as gender, age, and needle sharing.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Since the majority of studies on NEP effectiveness have relied on self-reported program attendance, these findings have important implications. Some of these studies have reported protective, but nonsignificant or equivocal, associations between NEP attendance and incidence rates of HIV, hepatitis B virus, and HCV (5, 12
, 23
, 24
). Furthermore, studies in Montreal and Vancouver, Canada, reported an elevated incidence of blood-borne infections among attendees versus nonattendees of NEP (9
11
). Although the majority of studies have shown that NEPs are associated with reduced incidence of blood-borne infections (1
), the above exceptions have generated considerable controversy (25
29
). A congressional ban on federal funds supporting NEPs in the United States has been upheld, in part because of concerns that some NEPs have not proven to be effective. Our data suggest that beyond the problem of selection bias that has been documented previously (1
, 11
, 15
, 30
), differential misclassification of self-reported NEP attendance among HIV seroconverters may have led to biased interpretations of NEP effectiveness in earlier studies.
The finding that HIV seroconverters were twice as likely to overreport NEP attendance is probably attributable to socially desirable responding. Among this subgroup, the majority of discrepant reports of NEP use took place after persons had first tested positive for HIV infection. Compared with the remainder of the cohort, persons newly diagnosed with HIV infection might have had a stronger desire to express protective or health-seeking behaviors to an interviewer. This finding is consistent with an earlier report from our group that found that HIV-seropositive participants were more likely to overreport syringe disinfection and condom use than were HIV-seronegative participants in an interviewer-administered assessment compared with ACASI (19).
After adjustment for HIV serostatus, overreporting of NEP use remained significantly associated with a decreased odds of needle sharing. This finding is in keeping with the hypothesis of socially desirable reporting, since IDUs who overreport NEP attendance may also be less likely to admit to a high-risk behavior such as sharing needles. In fact, at each semiannual study visit, participants are provided with risk reduction counseling, which may potentiate the tendency for underreporting such behaviors. On the other hand, overreporting of NEP attendance was also associated with more frequent injection. A previous report by Mensch and Kandel (31) found that IDUs who acknowledged injecting illicit drugs were not more likely to underestimate injection frequency. Hence, the higher injection frequencies observed among overreporters in our study is consistent with their higher risk profile for HIV seroconversion. The extent to which these data reflect the socially desirable response dimensions of "self-deception" or "impression management" as described earlier by Latkin and Vlahov (32
) requires further study.
In light of our findings, a possible method to reduce the misclassification we observed may be to incorporate ACASI in studies that evaluate the impact of NEP on the incidence of HIV and other blood-borne infections. Conceivably, ACASI could obtain more valid data on protective behaviors such as NEP attendance, beyond its well known ability to capture higher reports of stigmatized behaviors such as needle sharing (17, 18
). Further studies using ACASI to evaluate the validity of NEP attendance would be valuable, although such methods would not serve to eliminate other recall problems. Our study also serves as a reminder to ensure that interviewers remain blinded to HIV serostatus in situations in which ACASI is not an option.
Interpretations of our study findings should take into consideration several limitations. The primary purpose of this analysis was to explore the biases that can result from misclassification of exposure assessment and how this can mask true underlying associations between NEP attendance and HIV seroconversion. A unique feature of this analysis was our ability to document and account for misclassification by linking data from two independent sources. This afforded the opportunity to examine self-reports and actual NEP visits as well as to account for "secondary exchange" (i.e., the practice of obtaining NEP needles for other IDUs). Although we attempted to adjust for secondary exchange by not classifying such cases as discrepant reports, we were unlikely to have entirely eliminated misclassification altogether because secondary exchange is common in Baltimore (33). However, such remaining misclassification would serve only to underestimate the extent to which self-reports of NEP attendance could, in turn, underestimate a negative association of NEP.
Another limitation of this study is that we could not pinpoint the exact date when HIV seroconverters became consciously aware of their HIV-seropositive diagnosis. Although study records meticulously recorded the dates when participants' blood was drawn and tested and when results were disclosed in cases in which a specimen initially tested HIV positive on ELISA and Western blot, it was customary to request that the participant consent to a redraw to confirm the first test. Therefore, confirmed HIV results could have been disclosed to persons a month or more after the date when they first tested positive, in which case the recall period for behaviors pertaining to HIV seroconversion could have spanned two semiannual visits (i.e., 1 year). To address this to the extent that was possible, we checked the time between the first positive visit and the first instance of overreport of NEP and found that the median duration was 170 days. This finding is consistent with the interpretation that the majority of the discrepant self-reports of NEP use among HIV seroconverters occurred after they first learned of their HIV-positive serostatus.
It is important to bear in mind that cohort participants were recruited as early as 1988 and that many had a high level of rapport with our interviewers. Although it may be difficult to assess for certain whether this decreased or increased the level of socially desirable responding compared with cross-sectional studies or those with shorter duration of follow-up, our findings are in accord with a report by Mensch and Kandel (31), in which they documented that once an IDU participant has decided to acknowledge participation (injecting drugs), interviewer familiarity does not appear to affect reported frequency of participation in the behavior as much as it does the unacceptable behaviors associated with the activity. Hence, in this analysis, familiarity with the interviewer may have increased levels of socially desirable responding. Nonetheless, caution should be exercised in generalizing our findings to other settings. However, we suspect that other prospective studies evaluating NEP effectiveness may be subject to similar levels of socially desirable responding.
From a health research perspective, it is encouraging that self-reports of NEP use among IDUs were accurate for the most part. However, from the perspective of evaluating an HIV prevention program, overreporting of NEP program use, particularly among high-risk persons who are at higher risk of HIV seroconversion, can seriously underestimate the negative association of NEP with HIV seroconversion, especially if the true association is relatively modest. Since reporting of NEP attendance appears to be prone to socially desirable responding, we recommend obtaining NEP attendance through program data or using ACASI. When this is not possible, it may be necessary to develop novel ways to minimize differential classification to ensure that the potential negative association of NEP are not significantly attenuated.
![]() |
ACKNOWLEDGMENTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The authors thank the entire NEP staff and Dr. Peter Beilenson, Commissioner of Health for Baltimore City.
![]() |
NOTES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|