1 University of Pennsylvania/VA Center for Studies of Addiction, Philadelphia, PA.
2 Laboratory of Epidemiology, New York Blood Center, New York, NY.
3 Research Triangle Institute, Washington, DC, and Research Triangle, NC.
4 Fred Hutchinson Cancer Research Center, Seattle, WA.
5 Abt Associates, Inc., Cambridge, MA, and Bethesda, MD.
6 University of Michigan, Ann Arbor, MI.
7 Harvard University School of Public Health, Cambridge, MA.
8 Previous address: National Institute of Allergy and Infectious Diseases, Bethesda, MD.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
assessment, risk; HIV; homosexuality, male; questionnaires; sex behavior; substance abuse, intravenous
Abbreviations: ACASI, audio computer-assisted self-interviewing; HIV, human immunodeficiency virus; IAQ, interviewer-assisted questionnaire; IDUs, injection drug users; VPS, Vaccine Preparedness Study.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
There are a number of challenges to the accurate assessment of HIV risk behaviors within the epidemiologic and prevention trial framework. Of particular concern is the potential for study participants to underreport episodes of viral exposure. This may occur as a result of embarrassment, privacy concerns, or fear of negative reactions, since sensitive behaviors must be assessed by research staff who are concurrently promoting risk reduction, either directly or indirectly (58
). Error may also result from the inability of study participants to recall potential exposures that occurred during the assessment interval, which is often 6 months or longer (9
). These reporting problems may be compounded by variations in the administration of interviews since prevention trials typically use multiple interviewers, often at multiple sites, with repeat assessments.
There is growing evidence that the measurement of behaviors, particularly those that are considered sensitive or socially undesirable, can be performed more effectively through the use of computer-assisted self-interview (10). Computer-assisted interviews without audio have been shown to be reliable and valid in both survey and clinical settings (11
13
). They have demonstrated acceptability among respondents, and there is substantial evidence that computer-assisted assessments may enhance accuracy in reporting sensitive information. (14
16
). The computer-assisted self-interview technology has been adapted to include sound (audio computer-assisted self-interviewing questionnaire (ACASI)). Using ACASI, the research subject listens to a recorded human voice and responds to the question via keyboard entry (or the use of another input device such as a "touch screen" or "mouse"). The addition of audio expands the utility of this method of interview administration by eliminating the requirement for respondent literacy. Equally important, the ACASI removes interviewer-introduced bias, standardizes question administration, and eliminates skip pattern errors. ACASI has been used in survey research and has been shown to enhance the reporting of behaviors that are often uncomfortable for individuals to disclose to others. For example, in a national survey of adolescent males, respondents who completed the questionnaire using ACASI reported engaging in risk behaviors (e.g., male-to-male sex, injection drug use, and sexual contact with drug injectors) at more than three times the rate of respondents who completed self-administered questionnaires (17
). Similar finding were reported when ACASI was tested in a survey of injection drug users (IDUs) visiting four syringe exchange programs (18
).
Given the importance of valid and reliable measurement of behaviors in studies of HIV infection, ACASI technology offers a potential to improve the quality of behavioral data and, consequently, our understanding of factors associated with HIV infection and the interventions that may prevent it. The Vaccine Preparedness Study (VPS) provided a unique opportunity to examine the equivalence of ACASI and interviewer-administered questionnaires (IAQ) within the framework of an ongoing longitudinal study of risk behaviors and HIV seroincidence (19, 20
).
The primary objective of this evaluation of ACASI was to test the hypothesis that individuals interviewed by this method would report equal or higher rates of participation in HIV risk behaviors compared with controls assessed via IAQ. Equivalence was hypothesized because of the context in which the ACASI was being tested. Our assumption was that the sensitivity of responding to questions about sexual behavior and drug use would be minimal among these cohorts that had already confidentially reported practicing behaviors that placed them at risk of HIV infection. The evaluation of the ACASI was introduced at the 12-month follow-up visit. Thus, study participants had been screened for HIV risk, found eligible, and enrolled, and they were previously interviewed two times by research staff who completed detailed assessments of HIV risk behavior. Equal and higher rates of reporting were pooled since these outcomes would support using this technology in future studies. Further, since a higher rate of reporting sensitive information is suggestive of increased accuracy, lower rates of reporting sensitive information would diminish the utility of this technology. Secondary objectives were focused on evaluating the feasibility of using ACASI in future studies and its acceptability among gay men and IDUs.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Study participants were drawn from six of the VPS sites: two IDU sites (New York City, New York, and Philadelphia, Pennsylvania) and four gay men's sites (New York City, New York; Chicago, Illinois; Seattle, Washington; and San Francisco, California). A total of 600 gay men (150 from each site) and 500 IDUs (250 from each site) were randomly selected for ACASI assessment. All subjects from these sites who were not selected for ACASI assessment served as controls. These sample sizes provided 95 percent power to reject the null within each risk group.
The IAQ version of the month 12 VPS questionnaire was used as the model for programming the ACASI. This questionnaire included nine demographic items, eight health questions, 31 questions about sexual activity, seven alcohol and noninjection drug use questions, and seven questions about injection drug use. Since many of these items had multiple questions incorporated within them, the questionnaire included more than 258 individual items and a complex skip pattern. The questionnaire was designed for administration in about 30 minutes by trained interviewers.
The ACASI version incorporated identical question sequence and skip patterns using a wording as similar as possible to those found in the IAQ version. Several items that required text responses were dropped from the ACASI, and some optional probes were not included. For estimation of the frequency of specific sexual activities during the follow-up interval, the ACASI offered assistance to participants who desired help in calculating the frequency. This help was modeled after the manner in which interviewers had been trained to provide such assistance by calculating a total frequency based upon estimates of frequency during a shorter time interval (e.g., per week) and then examining the total and recalculating if warranted. The ACASI interviews were completed using desktop computers located in private areas of the study sites.
At the data collection visit, a standardized administration protocol was implemented. After confirmation of group assignment, this protocol required trained staff to provide participants with a brief orientation to the system and to complete the registration of the participant by entering the subject's study identification and date of administration. All participants then completed five practice questions with staff present. These practice questions were designed to provide examples of each type of response (continuous, categorical, dichotomous) contained in the interview. After successful completion of the practice questions, the participant was left alone to complete the assessment and was told to contact staff who remained easily accessible in case of questions or problems. Research staff were instructed to remain available but not to directly observe the respondent during the ACASI completion.
Data were saved on the hard drive of the computer as the respondent moved from one section of the questionnaire to another. At the end of the interview, the data were saved onto a diskette, and all data from the hard drive were copied onto tape backups at the close of each day in which data had been collected. Data from all sites were regularly transferred via modem to a central database.
At the close of the ACASI interview, subjects were asked to complete a two-page, self-administered acceptability questionnaire, which asked about problems they might have encountered in using the ACASI. The acceptability questionnaire also asked respondents to indicate which mode of questionnaire administration they would prefer for future assessments, which mode they felt best protected the privacy of their responses, and which mode would elicit the most honest responses.
To test the hypothesis of equivalence, responses from subjects assigned to the ACASI condition were compared with responses from controls who completed the questionnaire via interviewer administration. For this test, a limited set of risk-related items were selected prior to study implementation. These items assessed behaviors considered to be the most sensitive to response biases and socially desirable reporting. For the gay men, the following questions were selected: 1) any unprotected receptive anal intercourse; and 2) sexual activity with any HIV-positive partner. For the IDUs, the questions were 1) injecting at a rate equal to or greater than one time per week; 2) any sharing of syringes; 3) use of a syringe after another person without cleaning it; and 4) not using the needle exchange program. As stated, equivalence between the two modes of administration was defined as ACASI reports of equal or greater frequency. Thus, the null hypotheses for these analyses was that the ACASI would yield significantly lower rates of reporting on these items. The null specified a minimum tolerable odds ratio of 0.80 and thus would be rejected if the odds ratio for the collection of key variables was greater than 0.80. Secondary objectives of feasibility and acceptability were assessed by descriptive analyses of participant responses to the acceptability questionnaire.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Testing equivalence
Responses to items selected to test the hypotheses of equivalence between ACASI and IAQ are presented in table 2 for the two risk groups. As can be seen, when compared with those interviewed via IAQ, significantly more of the sexually active gay men assigned to the ACASI condition (n = 510) reported having engaged in sex with men who they knew were HIV positive. In addition, a higher, although nonsignificant, rate of unprotected receptive anal intercourse was reported by those assigned to the ACASI assessment.
|
Thus, the null, that responses to the ACASI would yield lower rates of reporting selected items, was rejected for the gay cohort. Given lower frequency of injection reported among the IDUs assessed by ACASI, the null could not be rejected for the IDU cohort. Additional analyses were conducted to explore more fully the detectable mode effects on a sample of both "sensitive" and "nonsensitive" items.
Mode effects among gay men
As shown in table 3, men assessed via ACASI (n = 528) reported less certainty about the HIV serostatus of their sexual partners. Significantly more men assessed via ACASI stated that they were unsure of the serostatus of their partner, and significantly fewer men assessed via ACASI reported having sex with men they were sure were HIV negative. Significantly more men assessed via ACASI reported being tested for HIV outside the study, and fewer men reported being willing to join a vaccine trial when a preventive vaccine becomes available.
|
Mode effects among IDUs
Similar to the mode effects observed among the gay men, most of the items we examined among IDUs (n = 418) yielded no differences among IDUs (table 4). These items included reports of illness or hospitalization, being tested for HIV outside the study, and willingness to enroll in an HIV vaccine trial. However, significantly fewer ACASI-assessed IDUs reported having health insurance, and significantly fewer reported any injection during the follow-up interval.
|
|
|
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
These data extend our understanding of the potential application of ACASI beyond its use in brief surveys in which respondents are assessed once. In surveys in which ACASI has been used, the absence of an interviewer has been shown to result in more frequent reports of participation in behaviors known to be socially unacceptable or undesirable (11, 12
). In our study, all participants had been selected and originally enrolled because they reported having engaged in high-risk sexual and drug use behaviors. Within this context, we anticipated that disclosure of all but the most risky behaviors would be less problematic, and thus, differences between mode of administration were less pronounced. These data cannot evaluate the impact of repeated ACASI assessments. Subjects were assessed only once. Thus, the contribution of these data lies in their ability to address the power of ACASI to improve measurement even after relationships with the research staff have been well established.
Among the gay men, our hypothesis was wholly supported. Both unprotected receptive anal intercourse and the number of HIV-positive partners were reported more frequently among those assessed via ACASI. For the IDUs, one of the four items (frequency of injection) did not conform to our hypothesis. For the three remaining itemsneedle sharing, needle exchange use, and using a needle after another person without cleaning itthe null was rejected. Importantly, the most risky of these behaviors, using a syringe after another person without cleaning it, was reported 2.4 times more often among the subjects assessed by ACASI.
The lower rate of reporting injection frequency of weekly or greater was not anticipated. Either subjects underreported their drug use to the computer or overreported it to interviewers. Although our data cannot resolve this question and further research is obviously warranted, several factors can be considered. First, this finding is consistent with the recently reported findings of a test of ACASI among IDUs attending syringe exchange programs, in that lower rates of injection were reported when assessments were performed using ACASI (18). In addition, drug injection is typically episodic, and since all IDUs were enrolled during a period of their active use, reductions in injection rates at subsequent follow-up visits are not unexpected. Thus, the question posed by our finding is not, "Why do the ACASI respondents report less injection?" Rather it is, "Which estimate of reduced injection is more accurate?" Again, given the context of this assessment, a longitudinal study among self-disclosed drug injectors, it is possible that some noninjecting participants felt that it was more desirable to report continued injection in order to meet expectations of the researchers and remain in the study. Given this scenario, it is plausible that the contextual pressure to report injection was reduced among those assessed via ACASI.
Other findings were quite consistent with our hypothesis of equivalence. Of the 18 additional items examined here, only one yielded a finding that could not be readily explained by the influence of social desirability. The significantly lower reported rate of health coverage among the IDUs assessed by ACASI was unexpected. However, since most of these participants were likely to be enrolled in publicly funded programs undergoing major changes in name and coverage, confusion is not unexpected. It is also possible that there is embarrassment about not having health insurance or having lost it, and the ACASI rate may reflect this stigma. The issue is one that cannot be resolved by these data alone and requires further investigation.
The data reported here remind us that it is the context of the assessment that determines the sensitivity of a particular question. For example, no differences in the rate of reporting were found for many of the items considered quite sensitive in other settingsmen having sex with men or visiting a shooting gallery. Conversely, behaviors typically not considered difficult to report in other contexts, such as getting tested for HIV, yielded significantly higher rates when assessed via ACASI among gay men in this study. As would be expected during a vaccine efficacy trial (to avoid unblinding), subjects had been asked to return to the research site for all HIV testing during the course of the study. Thus, participation in the VPS can be seen as raising the sensitivity of disclosing HIV testing at other locations. ACASI may therefore be particularly useful in monitoring adherence to study expectations during actual trials.
Responses to the acceptability questionnaires provide further evidence of the feasibility of using this technology in studies of IDUs and gay men. All of these participants had been interviewed by study staff at two prior assessments and thus had experienced the assessment via interviewer administration. Those assigned to the ACASI had few problems in using it. Importantly, all of the problems identifiedspeed of administration, understanding the questions, and ease of making and changing entriesare able to be resolved via programming modifications. When asked which mode of administration they would prefer for future assessments, only a small proportion reported a preference for the IAQ. Importantly, the ACASI was strongly endorsed by respondents from both risk groups as more likely to elicit honest responses.
The use of ACASI technology has the advantage of true standardization of questionnaire administration, a primary objective of epidemiologic research and clinical trials. When IAQ are used, the attainment of such standardization requires a significant amount of training and ongoing supervision. In ACASI applications, questions, probes, and branching are prerecorded and preprogrammed so that all subjects with similar behavioral profiles hear the same voice asking the same questions in the same sequence. Thus, variations in administration caused by the interviewer are eliminated.
Together, these data suggest that the application of ACASI technology in studies of gay men and IDU is both feasible and acceptable. These data also suggest that ACASI improves the measurement of sensitive risk behaviors. In so doing, ACASI provides a useful tool to more accurately understand the behavioral factors associated with HIV transmission and the efficacy of preventive interventions.
![]() |
ACKNOWLEDGMENTS |
---|
The following institutions and persons associated with the HIV Network for Prevention Trials (HIVNET) participated in the VPS Protocol Team: Domestic Master Contractor, Abt Associates, Inc.: Drs. G. Seage and M. Gross; Statistical and Clinical Coordinating Center, Fred Hutchinson Cancer Research Center and University of Washington: Drs. T. Fleming and S. Self; Central Laboratory, Viral and Rickettsial Disease Laboratory, California Department of Health Services: Drs. H. Sheppard and M. Ascher; Repository Contractor, Biomedical Research, Inc.: Dr. J. Leff; Denver Department of Public Health and Hospitals: Dr. F. Judson; Fenway Community Health Center: Dr. K. Mayer; Howard Brown Health Center: Dr. D. McKirnan; New York Blood Center: Drs. C. Stevens and B. Koblin; New York University Medical Center: Drs. M. Marmor and S. Titus; Beth Israel Medical Center, New York: Dr. D. Des Jarlais; San Francisco Department of Public Health: Drs. S. Buchbinder; University of Pennsylvania: Drs. G. Woody and D. Metzger; University of Washington: Dr. C. Celum; National Institute of Allergy and Infectious Diseases: Dr. R. Hoff, M. McCauley, and Dr. Z. Rosenberg.
The authors are particularly grateful to the study coordinators at the participating sites: Howard Brown Health Center: Rashad Burgess; New York Blood Center: Francesca Valenti; New York Medical Center: Dr. Steve Titus; Research Triangle Institute: Tim Smith and Roman Pendzich; San Francisco Department of Public Health: David Colbert; University of Pennsylvania: Helen Navaline; University of Washington: Cheryl Malinski.
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|