Invited Commentary: Considering Bias in the Assessment of Respiratory Symptoms among Residents of Lower Manhattan following the Events of September 11, 2001

David Vlahov and Sandro Galea

From the Center for Urban Epidemiologic Studies, New York Academy of Medicine, New York, NY

Correspondence to Dr. David Vlahov, Center for Urban Epidemiologic Studies, New York Academy of Medicine, 1216 Fifth Avenue, New York, NY 10029 (e-mail: dvlahov{at}nyam.org).

Received for publication January 13, 2005. Accepted for publication April 1, 2005.


    INTRODUCTION
 TOP
 INTRODUCTION
 BIAS
 EXPOSURE AND OUTCOME ASSESSMENT
 POTENTIAL CONFOUNDING
 CONCLUSION
 References
 
The report by Lin et al. (1Go) in this issue of the Journal presents data on past-year and past-month respiratory symptoms among residents of Lower Manhattan after the September 11, 2001, attack on and collapse of the World Trade Center, as compared with residents of Manhattan's Upper West Side. Residents of both areas were surveyed 8–16 months after the attack. The recorded frequency of symptoms after September 11 was higher in the exposed area than in the unexposed (or less exposed) area. The pervasiveness of the smoke, dust, and debris that permeated parts of Lower Manhattan for months after the September 11 attacks makes the respiratory tract irritation and symptoms reported by the residents in this study entirely plausible. While respiratory irritation and symptoms have been described in firefighters who were heavily exposed during these events (2Go), this study is noteworthy and laudable for its attempt to focus on the general population. However, this study in some ways raises more questions than it answers. In particular, key questions about potential bias, assessment of exposure and health outcomes, and potential unmeasured confounding pose considerable challenges to the interpretation of the results documented here.


    BIAS
 TOP
 INTRODUCTION
 BIAS
 EXPOSURE AND OUTCOME ASSESSMENT
 POTENTIAL CONFOUNDING
 CONCLUSION
 References
 
Selection, reporting, recall, and attribution bias may affect estimates of observed associations between respiratory symptoms and proximity to the World Trade Center collapse. A central issue in most epidemiologic assessments involving voluntary response is whether persons included in the assessment reflect the sampling frame of interest and, particularly in studies such as this one, whether selection was differential between the case (exposed) and control areas. Lin et al. (1Go) report a response rate below 25 percent in both the exposed area and the less exposed area in Manhattan. The low response rate probably reflects both a general disinclination to answer surveys among New York City residents and competing sources for participant attention, especially in Lower Manhattan. Although this response rate is low, it is well within the range of response rates for mail surveys as reported in the literature (3Go), and in and of itself is not a cause for concern. The key question is whether the persons recruited into the study were systematically different from the sampling frame they were intended to represent. In an effort to increase response rates and assess whether a higher response rate was associated with different findings than the main ones documented here, the authors extended their recruitment efforts with more intense procedures put into place within subsets of the sampling frames for Lower Manhattan and the Upper West Side comparison area. These additional efforts went beyond the initial mailings and included repeat mailings and hand delivery of letters, posters, brochures, meetings in co-ops, and so forth. It is reassuring that after these intensive efforts, the response rates were elevated equally in the exposed and less exposed areas and that the symptom associations were unchanged. The authors suggest that this is evidence against bias in the sample selection. However, in order for this to be the case, it would have to be ascertained both that these extra activities were performed with equal intensity in all housing units (i.e., that there was no selection bias) and that there was no differential motivation to participate by residents at the exposed and control sites (i.e., that there was no reporting bias).

The first of these two biases presumably was minimized by the study design, and there is no indication that differential efforts were made at the case or control sites that would suggest differential selection bias. However, the concern about reporting bias is more difficult to allay. Given a study of respiratory consequences of the September 11 attacks (we presume that the motivation of the study was explained to some extent to potential participants), it remains plausible (and perhaps likely) that a primary incentive for participation in Lower Manhattan was a desire to validate respiratory symptoms, while different motives altogether (an altruistic desire to contribute to research in general, the incentive offered) may have been motivating participation in the control area. Therefore, although the demonstration of an association between event exposure and respiratory symptoms in the housing units where more intense recruitment effort was expended is reassuring, it does not dispel the concern that the association demonstrated here is biased. If persons who participated in the study in the case community were more likely to report respiratory symptoms, then the association demonstrated here could be an overestimate of the true association between living in Lower Manhattan and respiratory symptoms.

Returning to potential selection bias, another concern pertains to the sampling frame itself and whether or not it adequately represents persons of interest for this study. This study had, as a sampling frame, persons who were residents of the exposed or control areas approximately 1 year after September 11, 2001. It excluded persons who were moved out of their residences and returned after January 1, 2002. However, it has been estimated that nearly all (97 percent) residents of Manhattan living south of Chambers Street and west of Broadway (which includes the Battery Park City area that was part of the sampling frame of this study) were displaced by the attacks, and that 10 percent of this group had not returned to their homes by June 2002 (4Go). It is plausible (and perhaps likely) that the persons most affected by the September 11 attacks would be less inclined to return to their exposed community. Hence, this study may well have undersampled persons who were exposed to the September 11 attacks and could have underestimated both the magnitude of the exposure and the effect measure describing the association between exposure to the attacks and respiratory symptoms in the general population. However, absent clearer enumeration of differences in the characteristics of the actual sampling frame on September 11 and at the time of the study, it is difficult to assess the influence of this potential bias.

In addition to concerns about selection and reporting bias, the interval for recall in the study instrument presents challenges for interpretation. Lin et al. (1Go) asked participants who were recruited 8–16 months after September 11 to recall whether they had had symptoms during the prior 12 months, and then during the past month. The authors noted that participants reported elevation of symptoms after September 11 as compared with before September 11. However, it is likely that there is substantial recall bias, with potentially unpredictable effects (5Go, 6Go), inherent in asking participants to compare current respiratory symptoms with symptoms a year before.

In particular, and more concerning, there is here an almost unavoidable attribution bias. Residents of Manhattan were talking about the September 11 attack and its possible consequences for months after the event. Therefore, it is to be expected that, when asked about the presence of respiratory symptoms before and after September 11, participants might have reported that symptoms had increased. Critically, it is plausible (and perhaps likely) that persons who were living in Lower Manhattan, where there was substantial public debate about the potential scope of environmental exposure from the event for many months after September 11, were more likely than residents of other areas of Manhattan to report increasing symptoms that they implicitly attributed to September 11 exposure when suitably prompted.


    EXPOSURE AND OUTCOME ASSESSMENT
 TOP
 INTRODUCTION
 BIAS
 EXPOSURE AND OUTCOME ASSESSMENT
 POTENTIAL CONFOUNDING
 CONCLUSION
 References
 
This study compared survey results from residents of Lower Manhattan (the exposed or case community) with residents of the Upper West Side (the unexposed or control community). However, neither the case group nor the control group may be construed as a genuine population-representative sample. Apparent overselection of residents in Battery Park City and Chinatown in the case group and limitation of selection to the Upper West Side in the control group suggest that the effect estimates documented here fundamentally reflect differences between two specific areas in Manhattan rather than between exposed persons and a general population sample of unexposed persons. Therefore, these findings, while suggestive, are limited in documenting the relation between disaster-event exposure and respiratory symptoms in the general population. In addition, it is important to recognize that these findings do not substitute for effect estimates that describe the relation between individual exposure and respiratory consequences. In effect, the exposure here is ecologic (living in an area where environmental exposure was more likely), and inferences about individual exposure must be drawn with caution.

Another issue relevant to exposure assessment relates to the decision to exclude residents of the control area who worked in the exposed area. As Lin et al. (1Go) suggest, participants in the control community may have been working in other heavily exposed areas downtown and potentially were just as exposed, or more exposed, to the smoke and debris than were participants in the case communities. However, residents of the exposed communities also could have been working in or outside of the exposed areas. Therefore, without exclusion of persons who lived and also worked in the exposed area, the results documented here are probably an overestimate of risk. A more complete categorization and comparison based on both work location and place of residence might have been a more appropriate assessment of individual exposure.

With respect to the reliability of health outcomes used in this study, several studies have shown that self-reporting is a limited way to classify respiratory symptoms (7Go) and that diagnosis of respiratory diseases is difficult even for health professionals (8Go). The focus here on milder respiratory symptoms, which may be more reliably obtained through self-report (9Go), is reassuring. In addition, the available data do not differentiate between increased use of over-the-counter medications and increased use of prescription medications. Increased use of the former would be a less reliable marker of respiratory disease, challenging the inference that can be drawn from these observations.


    POTENTIAL CONFOUNDING
 TOP
 INTRODUCTION
 BIAS
 EXPOSURE AND OUTCOME ASSESSMENT
 POTENTIAL CONFOUNDING
 CONCLUSION
 References
 
Lin et al. (1Go) attribute the symptoms documented to September 11-related environmental exposures. However, apart from the methodological issues discussed above, there are other potentially important explanations for the documented symptom changes. During the same time frame, studies of the mental health response in the general population of New York City showed elevated levels of anxiety, posttraumatic distress, and depression, and rates were higher in Lower Manhattan than elsewhere on the island (10Go). The association between stress responses and respiratory symptoms has long been appreciated (11Go–13Go), but Lin et al. cannot report on the extent to which stress reactions may have contributed to the respiratory symptoms documented here. As Lin et al. noted, two reports from Manhattan published after September 11 showed an association between psychological distress and respiratory symptoms (14Go, 15Go). We acknowledge that disentangling the contributions of physical irritants and psychological distress to self-reported respiratory symptoms is challenging, but we suggest that psychosomatic interpretations of these findings must be considered as potential contributors to the changes in respiratory symptomatology documented here. Reassuringly in this regard, one study showed a decline in symptoms of psychopathology during the first year after September 11, 2001 (16Go), and, as such, sustained increases in respiratory symptoms in Lower Manhattan compared with other areas of Manhattan may preferentially implicate the smoke, dust, and debris as having an ongoing effect.


    CONCLUSION
 TOP
 INTRODUCTION
 BIAS
 EXPOSURE AND OUTCOME ASSESSMENT
 POTENTIAL CONFOUNDING
 CONCLUSION
 References
 
Ultimately, the impact of environmental exposures arising from the events of September 11 on long-term respiratory symptoms, function, and disease remains to be determined. Although we commend Lin et al. (1Go) on their efforts to collect unbiased data in a timely fashion to help address this issue and fully recognize that the choices made may have been the only ones practical and feasible, concerns about the methodological limitations of this study make it difficult to draw clear conclusions from the observations made here. Guidance from the literature is difficult because, as the report notes, the composition of the exposure is complex. Data on Gulf War veterans exposed to oil fires and desert conditions showed an increase in respiratory symptoms but no change in respiratory function (17Go, 18Go). Lin et al. note that a separate report including measures of respiratory function is forthcoming, and this will be important for further interpretation of these data. Long-term data on respiratory symptoms in populations studied in the aftermath of other disasters are mixed. A number of reports have assessed people exposed to volcanic eruptions; while symptoms of respiratory irritation are noteworthy after the event, many of the reports show no long-term effects (19Go, 20Go). Analogous events provide an imperfect comparison. Carefully designed longitudinal studies will be needed to identify possible longer-term effects of the environmental exposures that accompany such disasters.


    ACKNOWLEDGMENTS
 
This work was supported in part by the National Institute of Mental Health and the National Institute on Drug Abuse.

Conflict of interest: none declared.


    References
 TOP
 INTRODUCTION
 BIAS
 EXPOSURE AND OUTCOME ASSESSMENT
 POTENTIAL CONFOUNDING
 CONCLUSION
 References
 

  1. Lin S, Reibman J, Bowers JA, et al. Upper respiratory symptoms and other health effects among residents living near the World Trade Center site after September 11, 2001. Am J Epidemiol 2005;162:500–7.
  2. Prezant DJ, Weiden M, Banauch GI, et al. Cough and bronchial responsiveness in firefighters at the World Trade Center site. N Engl J Med 2002;347:806–15.[Abstract/Free Full Text]
  3. Sheehan K. E-mail survey response rates: a review. J Comput Mediat Commun 2001;6(2). (Electronic article). (http://jcmc.indiana.edu/vol6/issue2/sheehan.html). (Accessed March 24, 2005).
  4. 9/11 United Services Group. A study of the ongoing needs of people affected by the World Trade Center disaster: key findings and recommendations. New York, NY: 9/11 United Services Group, 2002. (http://www.9-11usg.org/uploadedFiles/Ongoing_Needs_Study_7_8_02.pdf). (Accessed March 24, 2005).
  5. Mathiowetz NA, Dipko SM. A comparison of response error by adolescents and adults: findings from a health care study. Med Care 2000;38:374–82.[CrossRef][ISI][Medline]
  6. Brogger J, Eagan T, Eide GE, et al. Bias in retrospective studies of trends in asthma incidence. Eur Respir J 2004;23:281–6.[Abstract/Free Full Text]
  7. Mannino DM, Gagnon RC, Petty TL, et al. Obstructive lung disease and low lung function in adults in the United States: data from the National Health and Nutrition Examination Survey, 1988–1994. Arch Intern Med 2000;160:1683–9.[Abstract/Free Full Text]
  8. Barr RG, Herbstman J, Speizer FE, et al. Validation of self-reported chronic obstructive pulmonary disease in a prospective cohort study of women. Am J Epidemiol 2002;155:965–71.[Abstract/Free Full Text]
  9. Asmussen L, Olson LM, Grant EN, et al. Reliability and validity of the Children's Health Survey for Asthma. Pediatrics 1999;104:e71. (Electronic article).[Abstract/Free Full Text]
  10. Galea S, Ahern J, Resnick H, et al. Psychological sequelae of the September terrorist attacks in New York City. N Engl J Med 2002;346:982–7.[Abstract/Free Full Text]
  11. Wright RJ, Rodriguez M, Cohen S. Review of psychosocial stress and asthma: an integrated biopsychosocial approach. Thorax 1998;53:1066–74.[Free Full Text]
  12. McFarlane AC, Atchison M, Rafalowicz E, et al. Physical symptoms in post-traumatic stress disorder. J Psychosom Res 1994;38:715–26.[CrossRef][ISI][Medline]
  13. Weisberg RB, Bruce SE, Machan JT, et al. Nonpsychiatric illness among primary care patients with trauma histories and posttraumatic stress disorder. Psychiatric Services 2002;53:848–54.[Abstract/Free Full Text]
  14. Fagan J, Galea S, Ahern J, et al. Self-reported increase in asthma severity after the September 11 attacks on the World Trade Center—Manhattan, New York, 2001. MMWR Morb Mortal Wkly Rep 2002;51:781–4.[Medline]
  15. Szema AM, Khedkar M, Maloney PF, et al. Clinical deterioration in pediatric asthmatic patients after September 11, 2001. J Allergy Clin Immunol 2004;113:420–6.[CrossRef][ISI][Medline]
  16. Galea S, Vlahov D, Resnick H, et al. Trends of probable post-traumatic stress disorder in New York City after the September 11 terrorist attacks. Am J Epidemiol 2003;158:514–24.[Abstract/Free Full Text]
  17. Karlinsky JB, Blanchard M, Alpern R, et al. Late prevalence of respiratory symptoms and pulmonary function abnormalities in Gulf War I veterans. Arch Intern Med 2004;164:2488–91.[Abstract/Free Full Text]
  18. Kelsall HL, Sim MR, Forbes AB, et al. Respiratory health status of Australian veterans of the 1991 Gulf War and the effects of exposure to oil fire smoke and dust storms. Thorax 2004;59:897–903.[Abstract/Free Full Text]
  19. Afane Ze E, Roche N, Atchou G, et al. Respiratory symptoms and peak expiratory flow in survivors of the Nyos disaster. Chest 1996;110:1278–81.[Abstract/Free Full Text]
  20. Bradshaw L, Fishwick D, Kemp T, et al. Under the volcano: fire, ash and asthma? N Z Med J 1997;110:90–1.[ISI][Medline]