1 Center for the Studies of Addiction, University of Pennsylvania, Philadelphia, PA.
2 INSERM Unité 330, Université Victor Segalen Bordeaux-2, Bordeaux, France.
Received for publication March 7, 2002; accepted for publication November 19, 2002.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
bias (epidemiology); follow-up studies; HIV; substance abuse, intravenous
Abbreviations: Abbreviation: HIV, human immunodeficiency virus.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Despite its importance, research on the topic of retention and its impact on study validity has been sparse. Among drug users at risk of HIV infection, very little is known regarding the correlation between study retention and salient participant characteristics. Although a few studies have examined loss to follow-up (1114), they did not recruit the injection drug users at highest risk of becoming infected with HIV. Furthermore, the timeliness of follow-up has rarely been examined. Only one study assessed factors associated with timeliness of follow-up among drug users (5) and describes methods used to achieve a 96.6 percent retention rate. It reported that only unemployment predicted difficulty in retention, but the authors did not analyze risk behaviors or drug-use behaviors in detail.
Given the potential for "retention biases" to alter estimates of seroincidence and risk behaviors, the careful analyses of retention have particular relevance to studies of populations at high risk of HIV infection. In this paper, we analyze both timeliness and loss to follow-up in a cohort of injection drug users whose high risk of HIV infection was documented by an incidence rate above two per 100 person-years. Our objective in these analyses is to identify sociodemographic and behavioral characteristics associated with timeliness and loss to follow-up, in order to know whether or not the results obtained with this cohort were likely to be biased, and to elicit the profile of subjects on whom additional retention efforts should be concentrated. In addition, by analyzing timeliness and loss to follow-up in parallel, we could know if variables associated with the former would be the same as those associated with the latter, in order to understand what the underlying processes could have in common.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Potential subjects were made aware of the study using targeted street outreach and outreach visits to local detoxification facilities. Subjects who expressed interest in participation were provided with information about the study and given an opportunity to have their questions about participation answered. Informed consent procedures were completed with all subjects prior to the collection of behavioral data and biologic specimens. To be eligible for enrollment, subjects had to be HIV seronegative, to have injected drugs at least three times weekly over the prior 3 months, to have obtained less than half of their needles or syringes from the exchange program during the prior 3 months, and to be either out of drug treatment or in methadone treatment for less than 6 months. Subjects not meeting these criteria could be enrolled if they reported using a needle or syringe at least two times after a person known to be HIV positive and/or after at least three persons of unknown HIV serostatus during the same time interval. Enrollment occurred on the second office visit following the delivery of HIV test results. A total of 263 injection drug users were thus enrolled between September 1997 and June 1998.
Retention procedure
A structured procedure was used to maximize the retention and timeliness at follow-up visits. At the completion of baseline enrollment visits, target dates for follow-up visits were scheduled for the closest weekday matching a 6-month follow-up interval. Two weeks prior to the scheduled follow-up visit, a reminder letter was sent to the participant. Reminder phone calls were made to participants 1 week prior to the follow-up visit and again 1 day prior to the visit. A visit could be rescheduled by the participant at any of these structured contacts or at any time the subject contacted the study site to request a change. Subjects who showed up for a visit on an unscheduled day were accommodated, provided staff were available and the visit occurred within 1 month before or anytime after the targeted date. When a visit failed to occur, the following steps were undertaken until contact with the participant was achieved. 1) Immediately after a missed visit, telephone contact was initiated and staff continued calling until contact with the participant was accomplished. 2) After 1 week, a letter was sent to all contact addresses provided by the participant (letters did not disclose the nature of the study and simply requested that the participant call the study office to schedule an appointment). 3) After 3 weeks, a letter was sent again, followed by at least two calls to the participant and his/her contacts, and an initial outreach visit was made. 4) Finally, starting at week 3, calls and outreach visits were made weekly, and letters were sent monthly. Staff visited shelters and food lines, leaving sealed letters for pick-up by the participant should he or she visit the program. Also, monthly checks of local and state correctional facility inmate registries were completed.
To assist in follow-up activities, the study utilized a computerized participant information system called "participant file." This software was designed and programmed by members of the research team, and it was used to manage follow-up procedures by providing secure, immediate access to a variety of useful follow-up information on each individual subject. All appointment dates and contact information were included in this database. Contact information, collected at each visit (prescreen, baseline, follow-up), included the following: 1) names, nicknames, aliases, phone numbers, and addresses of the subject, his/her mother (including maiden name), two or more people knowledgeable on how to contact the subject, other study participants known by the subject, and people who would be contacted in case of incarceration; 2) names and locations of shelters where the subjects had stayed; churches, missions, and soup kitchens frequented by the subject; daytime and nighttime "hang-outs"; medical care contacts and treatment programs; and probation or parole offices. Whenever the subjects home address and "other contacts" locator information were found to be no longer valid, the following procedure was implemented: Facilities and venues where the subject might seek shelter were contacted; all shelters where the subject had reported that he/she stayed in the past were visited by the study team; to determine if an out-of-contact subject was incarcerated, the Record Room of the Philadelphia County prisons was contacted to inquire by name and/or alias of the subject, and the state and federal prison websites (http://www.cor.state.pa.us and http://www.prisons.com/doclist) were browsed to search by name, date of birth, and Social Security number. All attempts to locate subjects were recorded in the "participant file." This allowed all the staff to know the status of any missed subject so that others could assist in follow-up work if necessary. Access to this encrypted data was strictly controlled through a database security system. The database was stored on a removable disk that was locked in a safe each night.
Data collection and analysis
At each visit, interviewer-administered questionnaires were used to collect detailed contact information, sociodemographic characteristics, alcohol and drug use frequency, injection practices, sources of needles, HIV-related sexual behavior, participation in drug treatment, and HIV status of injection partners and sexual partners. All subjects received enhanced HIV counseling and were tested for their HIV status at each visit. For the analyses presented here, home addresses were used to compute the distance to the study center, using ArcView GIS software (15). The date initially scheduled for the 6-month visit and the actual day of the visit were used to create a "timeliness" variable that classified subjects as 1) "early" (2 or more days in advance), 2) "on time" (from 1 day early to 1 day late), 3) "4 weeks late" (228 days late), and 4) ">4 weeks late" (at least 29 days late) (table ). Because we were interested in detecting trends across these groups, we compared them using the Mantel chi-square test for trends (categorical data) and the Spearman nonparametric correlation test (continuous data) (7, 1618); multivariate analysis was performed using polytomous logistic regression (19). Subjects who did not complete a 6-month visit within a year after enrollment and were not known to have died were considered as lost to follow-up, which formed the retention variable for these analyses. Those classified as lost to follow-up were compared with those retained using the Pearson chi-square test (categorical data) and the Kruskal-Wallis test (continuous data), and multivariate analysis was performed using dichotomous logistic regression (7, 1618). Logistic regressions, either polytomous or dichotomous, were performed to assess the effect of each variable, controlling for confounding by other variables. Variable selection was done according to the following strategy. Four groups of variables were defined: 1) sociodemographic and geographic characteristics, 2) drug-use variables, 3) drug-related risk factors and treatment characteristics, and 4) variables describing sexual activity. For each group, a logistic model was built, starting with all variables with a p value below 0.25 in bivariate analysis and eliminating step-by-step those variables with the highest p value, until none of them had a p value above 0.15. Variables of each of these models were then put together in a common model, from which a final model was built by eliminating variables with the highest p value, regardless of their group of origin, until none of them had a p value above 0.10. This strategy was applied for the timeliness and the retention multivariate analyses separately, resulting in one final model per outcome of interest. All statistical analyses were conducted using SPSS software (SPSS, Inc., Chicago, Illinois).
|
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Bivariate analysis
Bivariate analyses indicated that the later subjects completed their visit, the more likely they were to be younger, to have higher income, to live farther from the study site, to have used hallucinogens during the past 6 months, to have a shorter history of drug use, and to have had a sex partner during the past 6 months. Further, these subjects were less likely to have spent a night in a shelter or welfare residence during the past year, to have used a needle after someone either known to be HIV positive or of unknown serostatus, to have shared rinse water, cotton, or cooker during the past 6 months, and to have gotten their needles at a needle exchange program (table ). Subjects lost to follow-up were more likely to have a high school diploma or equivalent, to have moved in the past year, and to have obtained needles in a shooting gallery or a drug store and less likely to have obtained them from a needle exchange program. However, the principal source of needles did not significantly differentiate subjects of different timeliness or retention status.
|
|
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The procedures that we implemented to increase the retention and timeliness of follow-up visits were similar to those described by other authors dealing with subjects from disenfranchised groups (15): Staff members were assigned specific roles and responsibilities; an automated tracking system was used; detailed contact information for each subject was collected; staff used multiple contact methods (mail, phone, outreach visits); staff conducted field outreach for subjects at shelters, food lines, and prisons; there was no limit to the number of attempted contacts, as long as the respondent did not explicitly refuse to participate; flexible appointment scheduling was implemented; participants were compensated for their travel and time at each visit (two public transportation tokens and $20); and personalized counseling sessions were used in which drug-related and health-related issues were addressed and referrals provided. In addition, our team has conducted epidemiologic studies with Philadelphia drug users since 1989. Further, we met regularly with the studys Community Advisory Board, seeking feedback and guidance on our methods.
In examining the characteristics of those retained, we failed to find any association between drug use and retention or timeliness. Hallucinogen use during the 6-month interval prior to enrollment was more frequent among those completing follow-up late, but this was not confirmed by the multivariate analysis. Such an absence of association is counterintuitive, because one could expect heavy drug use to have some influence on a subjects likelihood of retention and timeliness. Drug use has been found to be associated with retention in methadone treatment, although inconsistently (2126), but not in HIV risk-related studies (5, 1114, 20).
Likewise, HIV risk-related behaviors were more likely to be associated with timeliness than with retention in this study. Those who reported injecting more than two times per week in a shooting gallery were more likely to be late, but this was not associated with loss to follow-up. In the univariate analysis, injection drug users who had used a needle after someone known to be HIV positive or of unknown serostatus were more likely to complete follow-up on time, but this relation did not hold in the multivariate analysis and was not found for retention. Although needle sharing was not associated with timeliness or retention, more frequent sharing of rinse water, cotton, or cookers was associated with completing follow-up early but not associated with loss to follow-up. Subjects who reported backloading during the 6 months prior to assessment were more likely to be late, but this behavior was not related to retention. With respect to sources of needles, those retained were significantly more likely to use the needle exchange, although follow-up contacts were never initiated through the needle exchange program, and those lost were significantly more likely to obtain needles in a shooting gallery. Thus, with the exception of sources of needles, none of the risk-related variables was associated with loss to follow-up.
By contrast, several sociodemographic and geographic variables were associated with timeliness or with retention. Subjects with less than a high school education and those without health insurance were more likely to be retained, while subjects with low income and those who had spent a night in a shelter were more likely to be on-time completers. Thus, the most destitute participants were neither the most difficult to retain nor the most difficult to assess on schedule. Geographic and sociodemographic variables are consistently found to be associated with retention in HIV risk-related studies of injection drug users, although they vary in nature. Housing instability was a significant factor of loss to follow-up in our study, and it is cited by others (11, 13, 14). Younger subjects appear to be more difficult to retain (11, 13); in our study, they were more difficult to assess on time but were not more likely to be lost. Likewise, living farther away from the study site, a factor for dropout elsewhere (12), was associated with timeliness in our cohort but not with loss.
In studying timeliness and retention in parallel, we found that these outcomes had no significant factor in common. This suggests that the underlying mechanisms are different; in other words, subjects lost to follow-up cannot be considered as "extremely late but potential completers."
The limitations of our study are threefold. First, the results found in Philadelphia may not generalize to other locations. Variation in the nature of geographic and sociodemographic variables associated with retention from one study to another, including ours, suggests that the phenomenon is driven by the population and site characteristics, study design, and/or specificity of the tracking and retention method and its implementation. Results might also vary with the magnitude of the loss to follow-up. Housing instability, however, seems to be a common factor across studies. Second, our analysis of retention factors may lack statistical power, because of the small number of subjects lost to follow-up. Obtaining a large sample of high-risk injection drug users is not an easy task: We had to screen 1,049 injection drug users to recruit 263 who met the inclusion criteria. Larger samples have been obtained elsewhere, but either annual HIV incidence rates were below 1.5 percent (11, 20) or the retention rate at 6 months was below 70 percent (27). To gain insight on our ability to detect the factors associated with retention, we computed the minimal detectable risk (18). With a type I error level of 0.05, a power of 0.80, and frequencies of exposure in the retained group ranging from 15 percent to 75 percent, we could detect risks between 1.66 and 1.90. Thus, in spite of the small number of subjects in one group, we could still detect factors having a moderate effect on retention. Third, this study was essentially observational; HIV pre- and posttest counseling was the sole intervention component. When reduced drug use was an intervention outcome, as is the case in methadone programs, intervention and retention levels were frequently correlated (2124, 26). This has not been found to be the case in HIV risk-related interventions (5, 1214). Finally, these analyses examined retention after 6 months of follow-up. Many longitudinal studies require involvement of participants for much longer periods of time. Future research will be required to assess the relation between the factors associated with short-term versus longer-term retention. However, it should be noted that loss to follow-up is generally considered highest during the first follow-up interval.
Bearing in mind these limitations, our analysis suggests that the results obtained in this follow-up study of injection drug users might be minimally biased, if at all. For a bias to occur, several conditions must be fulfilled. In studies where an exposure-disease association is under investigation, attrition must be associated with both the disease (e.g., HIV infection) and the exposure (e.g., behavior, intervention, vaccine). Our study and several others (5, 1214) failed to find an association between HIV risk-related behaviors (exposure) and retention. Even if attrition were associated with disease and exposure, it would not necessarily bias the results, this being contingent upon the parameter of interest. For example, odds ratio estimates remain unbiased even when attrition over- or underselects exposed individuals, provided that it does not unbalance the ratio of the diseased over the nondiseased across exposure groups (8, 9, 28). More generally, the missing process is considered insignificant whenever it is unrelated to the outcome at any time, in which case standard statistical approaches will give valid inferences; statistical approaches will still give valid inferences when the missing process is related to the (observed) outcome at visits previous to dropout, provided that 1) this process is not related to the (unobserved) outcome at the missed visit, and 2) the analysis is conducted via a maximum likelihood procedure and appropriate modeling (29, 30). Some authors have concluded that, in multivariate analysis, the effect of attrition is absorbed by the fixed effects or affects only the estimates of the intercept (31).
In conclusion, the data presented here suggest that, when adequate efforts are invested in retaining drug users at risk of HIV infection, loss to follow-up can be minimized as a source of bias. Under these conditions, researchers should not refrain from assembling cohorts of drug users to study HIV-related issues, even among those subjects at highest risk.
![]() |
ACKNOWLEDGMENTS |
---|
The authors thank Roseanne Scotti for documentation and Robert Gross and Kevin Lynch for help with the manuscript.
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|