1 Epidemic Intelligence Service, Division of Applied Public Health Training, Epidemiology Program Office, Centers for Disease Control and Prevention, Atlanta, GA.
2 Vaccine Safety and Development Activity, Division of Epidemiology and Surveillance, National Immunization Program, Centers for Disease Control and Prevention, Atlanta, GA.
3 Division of Data Management, National Immunization Program, Centers for Disease Control and Prevention, Atlanta, GA.
4 Child Vaccine Preventable Disease Branch, Division of Epidemiology and Surveillance, National Immunization Program, Centers for Disease Control and Prevention, Atlanta, GA.
5 Center for Biologics Evaluation and Research, Food and Drug Administration, Rockville, MD.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
intussusception, rotavirus; vaccines
Abbreviations: AIC, Akaike Information Criterion; CDC, Centers for Disease Control and Prevention; CI, confidence interval; MCO, managed care organization; RV, rotavirus vaccination; VAERS, Vaccine Adverse Event Reporting System
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Although the reporting completeness of VAERS has been evaluated for some specific vaccine-event associations, this information cannot be generalized (4, 8
). Thus, the total vaccinated population that experiences an adverse event after a specific vaccine cannot be estimated with VAERS data alone. Such a population estimate is requisite to evaluate the strength of an association between a vaccine and an adverse event.
The capture-recapture methodology was designed to estimate population sizes on the basis of the proportion of subjects (re-)captured by two or more sources. It relies on four basic assumptions. First, the population should be closed or should not change in composition between the times of capture by the various sources. Second, sufficient information should be available in each source to match subjects from different sources in a unique manner. Third, the sources should be independent, i.e., capture by one source should not affect a subject's likelihood of being captured by another source. Finally, each subject should have an equal likelihood of capture, or certain segments of the population should not be more likely than others to be captured (9). Capture-recapture has been applied to a wide range of epidemiologic fields (10
17
) since first being introduced in epidemiology by Wittes and Sidel (10
) and Wittes et al. (11
). We studied how the capture-recapture methodology could be used to assess the reporting completeness of VAERS for intussusception after rotavirus vaccination (RV) and how it would allow the risk for this event to be estimated.
![]() |
MATERIALS AND METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
RV and intussusception
The tetravalent, rhesus-based rotavirus vaccine became the first licensed rotavirus vaccine in the United States in August 1998, and was subsequently introduced to the recommended routine childhood immunization schedule as a three-dose series at 2, 4, and 6 months of age (18). Intussusception (prolapse of a section of bowel into a more distal section, resulting in bowel obstruction) had been observed among vaccinated children in prelicensure trials of rotavirus vaccines (19
). Increased rates of intussusception among vaccinated children in postlicensure follow-up at Northern California Kaiser Permanente, as well as the receipt of 15 reports of intussusception after RV by VAERS, led to the suspension of the RV recommendation in July 1999 (20
, 21
). To assess more precisely the risk of intussusception after RV administration, the CDC initiated two in-depth epidemiologic studies, a multistate case-control study and a population-based retrospective cohort study. In addition, medical records of each case reported to VAERS were reviewed by CDC investigators to confirm the reported vaccination status and diagnosis (20
). Data from the two epidemiologic studies were used to assess the feasibility of utilizing the capture-recapture method to enhance vaccine safety surveillance in VAERS.
Intussusception and RV studies
The case-control study identified 429 cases of intussusception that occurred from November 1, 1998, to June 30, 1999, in selected areas of 19 states in the United States (22). The study was restricted to children aged 111 months. Cases were ascertained through review of hospital discharge and radiology records, followed by confirmation through medical record review. They were matched to 1,763 controls on date of birth and birth hospital. Vaccination history was obtained through telephone interviews with parents and health care providers. The study found an increased risk of intussusception in the 3- to 7-day and 8- to 14-day intervals after receipt of RV, which led to the withdrawal of RV from the recommended childhood immunization schedule (23
).
The retrospective cohort study identified cases that occurred between December 1, 1998, and July 16, 1999, in a cohort of 463,000 children from 10 managed care organizations (MCOs) nationwide (24). This study also included children aged 111 months and used the same method of case ascertainment and confirmation as the case-control study. Vaccination status was obtained from individual automated vaccination records and was confirmed through medical record review. The findings of increased risk were consistent with those of the case-control study (24
).
Capture-recapture inclusion criteria and matching
We selected all cases of intussusception after RV confirmed by charts from the computerized records of VAERS and the two studies that occurred in the common time frame (December 1, 1998, to June 30, 1999) and geographic area (the 19 states of the case-control study). We matched these on the following five variables: date of birth, state of residence, date of vaccination, date of diagnosis, and gender. To allow for recall bias in the VAERS reports, we allowed the reported vaccination and diagnosis dates to differ by up to 7 days from those found in the two studies. To further allow for coding or transcription errors on any of the five matching variables, we required only four of the five variables to match.
Capture-recapture analysis
We first computed Chapman estimates of the true number of cases of intussusception after RV by pairwise matching of the three sources (25). Let na denote the number of cases captured in data source A, nb denote the number of cases captured in data source B, and nab denote the number of cases captured in both sources. The Chapman estimate of the true number of cases is then calculated as N = [{(na + 1) x (nb + 1)}/nab + 1)] - 1 (26
).
We subsequently used log-linear models to estimate the true number of cases of intussusception after RV by using all three sources simultaneously. Unlike Chapman estimates, these models reveal the degree of dependence among the data sources. We constructed eight models to account for all possible two-source dependencies. From these, we selected the model with the lowest Akaike Information Criterion (AIC) score. This score is calculated by subtracting twice the number of degrees of freedom from the likelihood ratio statistic (G2). The AIC score thus favors the model with the lowest likelihood ratio statistic, while penalizing for an increasing number of parameters in the model.
To evaluate the impact of the duration between vaccination and the event (onset interval) and of severity of disease on ascertainment, we fitted separate models to subsets of data, defined by different levels of onset interval duration and severity. We then recalculated the capture-recapture estimate of the total number of cases by adding the estimates for each subset. The onset interval was divided into the following categories, as used in the in-depth studies of intussusception after RV: 02, 37, 814, and more than 14 days. Severity was defined by whether or not a case had resulted in surgery.
Goodness-of-fit-based confidence intervals were calculated for all capture-recapture estimates (27). The goodness-of-fit method is based on the likelihood ratio statistic, and the resulting intervals are thus asymmetric around N. This method of calculating the confidence intervals has the advantage of providing a lower limit that is higher than the total number of all observed cases (25
). Completeness of VAERS reporting was calculated as the ratio of the number of VAERS reports to the model estimate of the total number of intussusception cases.
Risk estimation
To evaluate the incidence rate of intussusception after RV for the previously established postvaccination risk intervals, we divided the estimated total number of cases that occurred in each interval by the total person-time for each interval. The total person-time was obtained by multiplying the total number of doses of RV, administered between December 1998, and June 1999, in the 19 states by the duration of each risk interval. The total number of first, second, and third doses administered for each state was estimated by multiplying the number of children in each state's birth cohort by dose-specific vaccination coverage rates in the state, as found among controls in the case-control study. Because the recommendations for RV allowed children up to age 7 months to receive the first dose, we assumed that children up to 5 months older than the recommended age were being vaccinated in the early months after the release of the vaccine (18). Under this assumption, an entire birth cohort was vaccinated during the 7 months of the study. The total exposed person-time is thus obtained by
19j = 1
3k = 1 Stj x Vacjk x duration, where Stj is the 1997 birth cohort of state j, Vacjk is the vaccination coverage rate of the kth dose among controls in the case-control study in state j, and duration is 5/365 and 7/365 years for the intervals 37 and 814 days, respectively.
Kramarz et al. (24) estimated the background rate of intussusception in unvaccinated children to be 25/100,000 person-years. Comparing the estimated incidence rates with this background rate, we obtained the relative risk of intussusception for a vaccinated child for the two risk intervals. To calculate the lower and upper confidence limits of the estimates of reporting completeness, incidence rates, and relative risks, we performed the above calculations on the lower and upper confidence limits of the capture-recapture estimates.
All analyses were performed by using SAS version 8 statistical software (SAS Institute, Inc., Cary, North Carolina).
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Chapman capture-recapture estimates
The Chapman estimate of the total number of cases was 104 (95 percent confidence interval (CI): 91, 127) for the pairwise analysis using VAERS and the case-control study, 78 (95 percent CI: 57, 208) for the pairwise analysis using VAERS and the cohort study, and 170 (95 percent CI: 98, 1,228) for the pairwise analysis using the case-control and cohort studies.
Log-linear modeling
The model with a single dependence term between the case-control and cohort studies had the lowest AIC and was thus selected as the best-fitting model (table 2). The total number of intussusception cases after RV as estimated by this model was 102 (95 percent CI: 91, 205). The reporting completeness of VAERS derived from this estimate was 47 percent (95 percent CI: 23, 53).
|
|
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
We estimated the relative risk of intussusception after RV to be 22.7 and 4.4 for the intervals 37 and 814 days after vaccination, respectively. The estimates from the case-control and cohort studies were 14.4 and 13.5 for the 37 days interval and 5.3 and 2.5 for the 814 days interval, respectively (22, 24
). Although our estimate for the first interval is larger than that of either study, the order of magnitude is comparable.
Our study was limited mostly by the last two of the four basic assumptions underlying capture-recapture (closed population, unique matching, independence of sources, and equal likelihood of capture). The population was closed because all sources captured subjects simultaneously. Matching was possible and unique, given the information available for the three sources. We estimated the likelihood of two distinct cases coincidentally matching by at least four of the matching variables (false-positive matching), given the total number of captured cases, to be only 0.04. Because of potential coding or reporting errors, we allowed one of the five matching variables to differ among matching cases. Had we not provided this possibility, four of the 35 matches would not have occurred. We confirmed all of these to be true matches, however, by identifying coding errors in the nonmatching variable for three matches (after comparison to the paper records) and by identifying an additional matching variable (date of receipt of another RV dose) for the fourth. In this fourth case, the nonmatching date differed by only one digit (05/06/98 vs. 05/16/98).
By using three sources, we only had to assume three-source independence, while using the log-linear models to test and possibly adjust for two-source interaction. We expected dependence to occur between the case-control and cohort studies, since cases of intussusception were ascertained in both studies through review of hospital discharge records. We expected cases ascertained by either source to be likely to also be ascertained by the other source (positive dependence). The denominator in the Chapman equation would then be artificially high and the capture-recapture estimate artificially low. The estimate of 170 derived from the pairwise comparison of these two studies was, however, higher than our final estimate, indicating a negative dependence. This may be because the case-control study drew most of its cases from large, predominantly public hospitals, whereas the cohort study drew cases only from MCOs.
Positive dependence could also have originated between VAERS and the case-control study if case identification by the case-control study led to an increased likelihood of being reported to VAERS. If this were true, the proportion of VAERS reports coming from the 19 states in the case-control study would have increased after the start of the study in July 1999. This proportion decreased, however, from 82 percent before July to 69 percent afterward. This suggests that the increase in VAERS reporting was caused by a general increased awareness, rather than just increased reporting among collaborating investigators of the case-control study. Similarly, dependence between VAERS and the cohort study could have originated if MCOs were more or less likely to report to VAERS. We could not explore this further because no information on MCO membership is collected in the VAERS reports.
Finally, we checked the equal likelihood of capture by stratifying our analyses by two variables known to affect ascertainment in VAERS: duration of the onset interval and severity of disease. Like Rosenthal and Chen (8), we found a slight decrease in the completeness of reporting with increasing interval between vaccination and the adverse event. We also observed a higher ascertainment rate by VAERS for severe cases compared with nonsevere cases. Stratifying the analyses on these variables hardly affected the total estimated number of cases, however. Unavailable variables that would have been of additional interest include the level of parental education and the type of health care provider or insurance.
The applicability of capture-recapture to epidemiology is still under debate. Cormack (28) went as far as to state that many capture-recapture studies lack the necessary information to give any reliable population size estimate. Although we followed the recommendations of Hook and Regal (29
), made in reply to Cormack's statement, we agree with the latter that our capture-recapture estimates should be interpreted with great caution, especially given the sparseness of some of the data. Our results, although not so different, are certainly not comparable in strength with the estimates obtained in the two epidemiologic studies used as sources for our analysis.
Some additional limitations of our study should be mentioned besides those inherent to the capture-recapture methodology. We based our estimated number of doses of RV on the vaccine coverage levels among controls in the case-control study. Because this study focused on areas of high coverage within the 19 states, we may have overestimated the number of doses and underestimated the relative risks of intussusception after RV. On the other hand, our estimate of 708,366 doses is lower than the total number of doses distributed by the manufacturer minus the number returned (1,156,813 for these 19 states) (P. Paradiso, Wyeth Lederle Vaccines, personal communication, 2000). In the unlikely event that all unreturned vaccines had been administered, the estimate of the relative risk of intussusception after RV would be 13.9 and 2.7 for the intervals 37 and 814 days, respectively.
We compared the incidence of intussusception in vaccinated children with the recent background estimate of intussusception in unvaccinated children. We judged this estimate to be the most accurate estimate available because it relied upon cases confirmed by charts. The estimated background rate is only half of an earlier estimate derived from hospital discharge records in New York, however (21). Using the earlier estimate as the background rate, we would have found our risk estimates to be exactly half of the ones reported.
Allegations of adverse events being caused by vaccination will occur more frequently in our world of increasing numbers of new vaccines, increasing vaccine coverage rates, and decreasing tolerance of adverse events. How applicable will our approach be for a different vaccine and a different adverse event? For our analysis of intussusception after RV, we were fortunate to have two additional sources of exposed cases available. Clearly, studies such as the case-control and cohort studies of intussusception after RV cannot be carried out to investigate every new concern of an association between an adverse event and a vaccine. In the context of increased automatization of medical records, however, complimentary sources for linkage to VAERS may be available, such as hospital discharge or clinic record databases. For denominator data in the incidence calculations, manufacturers' distribution data can be used to estimate the number of vaccines administered. A capture-recapture analysis may then be considered to evaluate the need of more costly case-control or cohort studies. In some regions of the world, automated databases are unlikely to be available, and the cost and effort of case-control or cohort studies may be prohibitive. Construction of ad hoc case registries through a sampled case-finding exercise to allow capture-recapture analyses may be considered as an alternative.
![]() |
ACKNOWLEDGMENTS |
---|
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|