a University of Zimbabwe, University of California at San Francisco Women's Health Program, Harare, Zimbabwe and the University of California at San Francisco, Department of Obstetrics, Gynecology, and Reproductive Science. Present affiliation: The Population Council, New York.
b University of California at San Francisco, Department of Epidemiology and Biostatistics.
c Program in Health and Behavior Measurement, Research Triangle Institute, Washington, DC.
Reprint requests to: Dr Janneke van de Wijgert, The Population Council, One Dag Hammarskjold Plaza, New York, NY 10017. E-mail: jvandewijgert{at}popcouncil.org
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods Zimbabwean women in three educational groups were surveyed about demographics and family planning using interviewer and ACASI modes. An exit survey was administered to elicit information about the participants' opinions and experiences using ACASI.
Results The majority of women (86%) preferred ACASI to interviewer mode. The reasons mentioned were always related to increased confidentiality and privacy. Ability to use ACASI and user preferences varied with educational level. More women with primary school or less education (53%) reported problems with computer use than women in the higher educational groups (1012%). The percentage of women having perfect response concordance between ACASI and interviewer modes increased significantly with education (64%, 81%, and 84% respectively; Ptrend < 0.001).
Conclusions Use of ACASI may be more feasible in Zimbabwe and other developing countries than was originally thought, but ACASI programs should continue to be improved and tested in various countries and population groups.
Keywords ACASI, reproductive health, survey methodology, Zimbabwe
Accepted 11 April 2000
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
A growing body of empirical data collected in the US indicates that interview methods that increase the privacy of the interview context can dramatically increase reports of sensitive and illegal behaviours.16 Based on the assumption that reports given in private are more honest, increased reporting of sensitive behaviours suggests that the reports are more accurate.16 Until recently, the primary means for such private interview methods has been paper self-administered questionnaires. However, the administration of paper self-administered questionnaires is restricted to populations where literacy approaches 100% and is therefore not feasible in most developing countries. Furthermore, paper self-administered questionnaires are not appropriate for surveys with complex designs and skip patterns.7
Recent studies indicate that interviews using a new technology, audio computer-assisted self-interviewing (ACASI), may also lead to increased reliability in sensitive reports.26 In ACASI, the survey questions are displayed on a laptop computer screen, and digitally recorded in any language the researcher desires. The quality of the sound is equivalent to that of a human voice. The participant reads the questions on the screen or listens to them through earphones and responds by typing the answers directly into the laptop computer. This method has several advantages: it ensures privacy for participants; does not require an ability to read or write; provides a standardized measurement system (limiting interviewer biases); is easily adapted to multilingual administration; and can incorporate complex skip patterning, branching, and consistency and range checking. Furthermore it creates an automatic data set allowing for immediate data management and analysis, and eliminates error often encountered during the data entry process.
Research on this new methodology in the US indicated that ACASI could be used without disrupting typical survey and research routines, and that virtually without exception respondents had no trouble using it.8,9 These results applied to multilingual contexts as well. Pilot studies found that it was possible to use ACASI to interview participants who spoke only Korean or Spanish, using field interviewers who spoke only English.10,11 The reliability of ACASI for collecting data on sensitive behaviours has been evaluated in two US national health surveys (the National Survey of Family Growth2,12 and the National Survey of Adolescent Males4), syringe-exchange programmes5, and family planning and STD clinics.6 In all surveys, participants who were assigned the ACASI mode reported significantly higher levels of sensitive information (including abortion history, unsafe sex, sharing of needles for injection drug use, and violent behaviour) than in an in-person interview.
Audio computer-assisted self-interviewing has not been used extensively in developing countries and only a few feasibility studies are currently ongoing or planned.13 This may be due to an assumption that people living in developing countries will not be able to use ACASI successfully. Most people in developing countries have limited formal education; limited or no experience of paid employment; limited access to television, newspapers and books; and very few have access to computers. Because of the potential advantage of ACASI over all other survey methods, and because very little is known about the feasibility of ACASI in a developing country setting, we conducted a pilot feasibility study of ACASI in Zimbabwe. The aims of this study were to determine whether Zimbabwean women in three different educational groups are able to use an ACASI program successfully, and to document their experiences. This study did not examine whether ACASI can improve the reliability of sensitive behaviour reporting in Zimbabwe.
![]() |
Methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Women in the lowest educational group were recruited from Nijo farm, which is a government-owned farm on the edge of Harare. The women who participated in the study were either farm workers themselves, or the wives of male farm workers. Women in the middle educational group were recruited from Spilhaus family planning and Epworth primary care clinics, located in Harare's high-density suburbs. Women in the highest educational group were college students recruited from the University of Zimbabwe (UZ), and Belvedere Teacher Training College. Both colleges are located in Harare.
Study procedures
The ACASI program was designed by researchers at the Research Triangle Institute in the US in conjunction with staff at the UZ-UCSF Women's Health Program, Harare, Zimbabwe. An IBM Thinkpad laptop computer with a black-and-white screen and attached earphones was used. The ACASI program included: automated range checks (i.e. the computer issued a warning when an out-of-range value was entered); the ability to repeat a question or go back to a previous question by pressing clearly labelled navigation keys; the ability to answer Don't know or Refuse to Answer by pressing clearly labelled keys; and the possibility of suspending and restarting the interview. The enter key was painted red, and clearly identified as such in the program's instructions. The ACASI program asked the participants to press the red key after typing a number (such as age), but would automatically advance to the next question after typing an answer to a yes/no or multiple choice question. Mistakes could be corrected by repeating or going back to a previous question using the above mentioned navigation keys; by re-entering a value after it was rejected by the computer's automated range check; or by asking the interviewer for help. All interviewers were thoroughly trained in the use of the ACASI program.
After the participant gave written informed consent, she was randomized into one of two study groups: women in the first group were interviewed by an interviewer first (interviewer mode), and then completed the same questions on a laptop computer using the ACASI program (ACASI mode). Women in the second group completed the questionnaire by ACASI first, and were then interviewed by an interviewer. In the interviewer mode, the interviewer read the questions as they appeared on the computer screen and entered the participant's responses. In the ACASI mode, the participant read the screen (if literate) and/or listened to the questions through earphones, and entered the answers into the computer herself. The interviewer left the room during ACASI mode after helping the participant with the first three practice questions. The time lag between ACASI and interviewer modes never exceeded 10 minutes.
The 17 survey questions in ACASI and interviewer modes were identical, and included nine demographic questions, seven questions about ever use of a variety of family planning methods, and one question about drug allergies. None of the questions were considered sensitive by our experienced research staff, which allowed us to document women's abilities and experiences using an ACASI program in the absence of potential severe under- or over-reporting due to the nature of the questions. After completion of the interviewer and ACASI procedures, an exit survey was administered, and travel reimbursements were dispensed. The exit survey was administered face-to-face by the interviewer, and consisted of 10 questions about the participants' experiences using ACASI.
Data analysis
Data were analysed using Stata 5.0 software.14 Demographic variables and responses to exit survey questions were cross-tabulated by educational group. Differences in proportions and trends were tested using the two-sided Fisher's exact test and the Cuzick non-parametric test for trend,15 respectively. A test-retest design was used to compare interviewer mode responses to ACASI mode responses. The responses obtained for each interview question by the mode of the interview were cross-tabulated and the measures of response agreement (Kappa for binary variables, intra-group correlation coefficient for categorical variables, and Pearson's correlation coefficient for continuous variables) were calculated to assess the extent of response concordance.
Ability to use ACASI was assessed by determining ACASI and interviewer mode response agreements in different educational and age groups. We assumed that response discordance in this study was mainly due to typing and other errors, rather than purposefully under- or over-reporting, because the ACASI program consisted of non-sensitive survey questions. Ability to use ACASI was also assessed by questioning the participants in an exit survey about their problems and experiences with ACASI, and by direct observation during the practice sessions.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Table 1 shows that women in the lowest educational group (n = 108) completed an average of 6 years of schooling (ranging from 07 years), compared to 11 years (812 years) in the middle (n = 111) and 16 years (1417 years) in the highest educational group (n = 67). Only 3% of the women in the lowest educational group had ever used a typewriter, computer, or automated teller (cash withdrawal) machine (ATM). These percentages were 43 and 96 for the middle and highest educational groups respectively (Ptrend < 0.01).
|
Table 1 shows the participants' opinions and experiences with ACASI by educational group. About half of the women with little education (53%) experienced one or more problems with computer use, as compared to 10 and 12% of the women in the other two educational groups (Ptrend < 0.01). The most frequently mentioned problem was difficulty with reading the computer screen due to illiteracy or low reading level (31 women) or bad eyesight (2 women). Problems with keyboard use were also reported and observed several times, despite the fact that all keys were clearly labelled. These problems included: putting too much pressure on the keyboard keys (15 women), not understanding how to use the navigation keys needed to move around the program and to correct mistakes (11 women), and forgetting to press the enter key after typing a number (6 women). Whether the participants preferred listening to the questions through earphones or reading them on the computer screen clearly depended on the level of education: 83% of the women in the lowest educational group preferred listening, whereas 70% of the students preferred reading (Ptrend < 0.01). While no one expressed any difficulty in understanding the spoken Shona text, 14 students thought that the computerized voice was tediously slow.
About one-third of the women in each educational group (38, 27 and 33%, respectively; Ptrend = 0.54) made at least one attempt to correct a mistake before completing the ACASI program. The majority of the women in the lowest (72%) and middle educational groups (79%) and all students who attempted to correct a mistake succeeded in doing so (Ptrend = 0.01). Mistakes were corrected by pressing the navigation keys to repeat or go back to a previous question (32 women); by typing over a previous answer that was not accepted by the ACASI program (23 women) or by calling in an interviewer to assist (4 women).
The majority of participants in all educational groups thought that multiple choice questions were the hardest to answer, and yes/no questions the easiest (Table 1). Multiple-choice questions were considered difficult because the categories were often thought to be confusing (mentioned by 36 women), especially when they did not contain the answer the participant had in mind (3). Many women also thought that it was time-consuming to read all the options (32); that it was difficult to remember all options when typing the final answer (8); and that it was easy to make a mistake (5). There were mixed feelings about questions that required typing a number. Some women complained that it was easy to make a mistake with the number questions, because the answer was not provided and too many different keys (including the red enter key) had to be pressed. Yes/no questions were generally considered to be straightforward.
To determine the response agreement between the two interviewing modes, ACASI mode responses were compared to interviewer mode responses using the Kappa coefficient for yes/no questions, the Pearson's correlation coefficient for number questions, and the intra-group correlation coefficient for multiple choice questions. The Kappa coefficients ranged from 0.742 to 0.891; the Pearson's correlation coefficients from 0.716 to 0.992; and the intra-group correlation coefficients from 0.619 to 0.704.
The ACASI and interviewer mode response agreement was also determined by educational group and age (Table 2). The ACASI and interviewer mode responses of 76% of the women matched perfectly; 20% had one discrepancy, and 4% had more than one discrepancy. The percentage of women with perfect concordance between the two interviewing modes increased significantly with education (65%, 82% and 84%, respectively; Ptrend < 0.01). It decreased with age, but not statistically significantly so (79%, 76% and 73%, respectively; Ptrend = 0.26).
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Our data also show, however, that the reliability of ACASI varied with the educational level of the respondent. Women with little education (primary school or less) had considerably more problems using the computer keyboard, reading the computer screen, and correcting mistakes than women in higher educational groups. The percentage of women with perfect agreement between ACASI and interviewer mode responses went up from 65% in the lowest educational group to 82% and 84% in the middle and highest educational groups. In contrast, there was a tendency for perfect response agreement to decrease with age, but not statistically significantly so. Older age could be associated with less familiarity with computers, less tendency to want to learn a new technology, and sub-optimal eye sight. Age and educational level were strongly correlated in this study (the students, in particular, were younger on average than the women in the other two groups), but the effect of educational level on ability to use ACASI was much more profound than the effect of age.
In this pilot study, we could not determine what minimum educational level would be required for optimal use of this particular ACASI program, and more work in that area is clearly needed. Furthermore, ACASI programs could be made more user-friendly for women with little education by considering the following suggestions: (1) Multiple-choice questions could be avoided as much as possible. Our data clearly show that most problems occurred with multiple choice questions; (2) The keyboard could be made more user-friendly. In this study, a regular laptop computer with a QWERTY keyboard was used. Keyboard options could be limited to responses (e.g. YES, NO, and numbers) and larger colour-coded keyboard keys could be used; (3) The text on the computer screen could be made larger. In this study, a 12 point text was used; (4) The mechanism to identify typing errors and correct mistakes could be improved. For example, a double-entry system could be build into the program. Lastly, designing different types of ACASI programs for different educational and other population groups could also be considered.
Women in the middle and highest educational groups who underwent the interviewer mode first were less likely to have perfect response agreement than women in the same educational groups who underwent ACASI first. The effect of the order in which procedures are carried out should be investigated further in future studies, and if this finding persists, explanations should be sought.
In this pilot study, a convenience sample drawn from women in three different educational groups was used. We therefore do not know whether the study results are generalizable to other women in the same educational groups. We do know, however, that the women in the middle educational group were similar in terms of demographic and contraceptive use characteristics to the women who generally participate in clinic-based studies in Harare.16 Furthermore, participation rates approached 100% in each educational group and we have no reason to believe that a systematic selection bias was present.
Designing an ACASI program can be time consuming and costly, particularly when questionnaires are lengthy, multiple translations are needed, or multiple population groups are targeted (for example, men and women, or different educational groups). However, ACASI eliminates the need for forms management and data entry, and greatly simplifies data management. It may therefore be cost-effective in large studies. More research is needed to determine the feasibility and cost-effectiveness of utilizing ACASI in large studies in developing countries.
Given the importance of collecting reliable survey data in reproductive health, the potential advantages of ACASI over all other survey methods, and the preliminary results we obtained in this study, we recommend that ACASI programs continue to be improved and tested for use in developing countries. Once country- and population-specific programs have been designed and tested, further research is also needed to determine whether they increase the reporting of sensitive behaviours, as they did in the US.
![]() |
Acknowledgments |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2 Duffer AP, Lessler JT, Weeks MF, Mosher WD. Impact of Incentives and Interviewing Modes: Results from the National Survey of Family Growth Cycle V Pretest. Conference Proceedings: Health Survey Methods. Hyattsville, MD: National Center for Health Statistics, 1996.
3
Bloom DE. Technology, experimentation, and the quality of survey data. Science 1998;280:84748.
4
Turner C, Ku L, Rogers SM et al. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science 1998;280:86773.
5 Des Jarlais DC, Paone D, Milliken J et al. Audio-computer interviewing to measure risk behaviour for HIV among injection drug users: a quasi-randomized trial. Lancet 1999;353:165761.[ISI][Medline]
6 Kissinger P, Rice J, Farley T et al. Application of computer-assisted interviews to sexual behavior research. Am J Epidemiol 1999;149: 95054.[Abstract]
7 Gribble J, Miller H, Rogers S. Interview mode and measurement of sexual behavior: methodological issues. J Sex Res 1999;36:1624.[ISI]
8 O'Reilly JM, Hubbard ML, Lessler JT, Biemer PP, Turner CF. Audio and Video Computer Assisted Self-Interviewing: preliminary tests of new technologies for data collection. J Official Statistics 1994;10: 197214.
9 Rogers SM, Miller HG, Forsyth BH, Smith TK, Turner CF. Audio-CASI: The Impact of Operational Characteristics on Data Quality. Joint Proceedings of the American Association for Public Opinion Research/American Statistical Association, Survey Methods Research Section, 1996.
10 Turner CF, Rogers SM, Hendershot TP, Miller HG, Thornberry JP. Improving representation of linguistic minorities in health surveys. Public Health Rep 1996;111:27678.[ISI][Medline]
11 Hendershot TP, Thornberry J, Rogers SM, Miller HG, Turner CF. Multicultural Audio-CASI: using English-speaking field interviewers to survey elderly Korean households. In: Warnecke RB (ed.). Health Survey Research Methods: Conference Proceedings. DHHS Pub. No. (PHS) 961013. Hyattsville, MD: National Center for Health Statistics, 1996.
12 Miller HG, Gribble JN, Mazade LC, Turner CF. Abortion and breast cancer risk: fact or artifact? Science of Self Report. Mahwah, NJ: Lawrence Erlbaum Associates, 1998.
13 Parish WL. National Institutes of Health abstract for Chinese health and family behavior research proposal (5R01HD34157). Bethesda, MD: National Institutes of Health CRISP on-line database, 1998.
14 Stata Statistical Software: Release 5.0. College Station, TX: Stata Corporation, 1997.
15 Cuzick J. A Wilcoxon-type test for trend. Stat Med 1985;4:8790.[ISI][Medline]
16 van de Wijgert JHHM, Mason PR, Gwanzura L et al. Intravaginal practices, vaginal flora disturbances, and acquisition of sexually transmitted diseases in Zimbabwean women. J Infect Dis 2000;181: 58794.[ISI][Medline]