Chlorpyrifos Induces Apoptosis in Rat Cortical Neurons that is Regulated by a Balance Between p38 and ERK/JNK MAP Kinases

Anne Caughlan*,1, Kathleen Newhouse*,1, Uk Namgung*,2 and Zhengui Xia*,{dagger},3

* Toxicology Program in the Department of Environmental and Occupational Health Sciences, and {dagger} The Department of Pharmacology, University of Washington, Seattle, Washington 98195-7234

3 To whom correspondence should be addressed at Department of Environmental and Occupational Health Sciences, Box 357243, 1705 Pacific Street, University of Washington, HSB, Room F561C, Seattle, WA 98195-7234. Fax: (206) 685-3990. E-mail: zxia{at}u.washington.edu.

Received August 22, 2003; accepted November 5, 2003

ABSTRACT

Chlorpyrifos, an acetylcholinesterase (AChE) inhibitor, is a widely used organophosphate pesticide. Recent concern has focused on its neurotoxicity that is not attributable to AChE inhibition. Here, we report that chlorpyrifos and chlorpyrifos-oxon, but not 3,5,6-trichloro-2-pyridinol (TCP; the breakdown product of chlorpyrifos and chlorpyrifos-oxon), induce apoptosis in primary cortical neurons cultured from embryonic day 17 or newborn rats. It is generally agreed that chlorpyrifos-oxon is approximately three orders of magnitude more potent than chlorpyrifos in inhibition of brain acetylcholinesterase activity. However, our data demonstrate that chlorpyrifos-oxon is only slightly more potent than chlorpyrifos in inducing apoptosis. This indicates that chlorpyrifos-induced apoptosis may occur independently of AChE inhibition, although AChE activity was not measured in this study. Furthermore, chlorpyrifos activates the ERK1/2 and p38 MAP kinases. Surprisingly, blocking ERK1/2 activation by the MEK inhibitor SL327 caused a small but statistically significant inhibition of apoptosis, while blocking p38 with SB202190 significantly accelerated apoptosis induced by chlorpyrifos. This suggests a pro- and anti-apoptotic role for ERK1/2 and p38, respectively. Although chlorpyrifos did not stimulate total JNK activity, it caused a sustained activation of a sub-pool of JNK in the nucleus and stimulated phosphorylation of c-Jun, a downstream target of JNK. Transient expression of a dominant negative c-Jun mutant inhibited chlorpyrifos-induced apoptosis, suggesting a role for JNK and JNK-mediated transcription in this cell death. Together, our data suggest apoptosis as a novel toxic endpoint of chlorpyrifos neurotoxicity in the brain that may be independent of AChE inhibition. Furthermore, activation of the ERK1/2 and JNK MAP kinases contributes to, while activation of the p38 MAP kinase counteracts chlorpyrifos-induced apoptosis in cortical neurons.

Key Words: chlorpyrifos; apoptosis; cortical neuron; CNS; MAP kinase; p38; c-Jun; ERK1/2; JNK; SAPK; SL327; SB202190.

In multicellular organisms, normal development and homeostasis in adults require a carefully regulated balance between pathways that stimulate cell proliferation and cell death. Apoptosis is a form of programmed cell death and inappropriate apoptosis may contribute to various neurodegenerative conditions including Parkinson's disease and Alzheimer's disease. Recent studies suggest that environmental toxicants including pesticides may contribute to the development of several neurodegenerative disorders including Parkinson's disease and amyotrophic lateral sclerosis (Nelson, 1995Go). Epidemiological studies have established an association between increased risk for Parkinson's disease and exposure to pesticides including the chlorpyrifos-family organophosphates (Checkoway and Nelson, 1999Go; Di Monte et al., 2002Go; Le Couteur et al., 1999Go). Furthermore, pesticide exposure impairs learning and memory in children (Guillette et al., 1998Go) and affects hippocampus-dependent spatial memory in rodents (Levin et al., 2001Go, 2002Go; Terry et al., 2003Go). In the developing brain, changes in the amount of apoptosis after exposure to environmental toxicants could cause structural changes that in turn lead to developmental deficits in brain function, including learning and memory. Hence, mechanistic studies of pesticide-induced neuronal apoptosis may provide valuable new information concerning the molecular basis of pesticide-induced apoptosis in neurons and the role of environmental toxicants in neurodegeneration. Furthermore, these studies may also provide insight regarding the impact of pesticide exposure on children's health and central nervous system (CNS) development, and suggest treatments to counteract the neurotoxicity of pesticides.

Chlorpyrifos (O, O-diethyl O-(3,5,6-trichloro-2-pyridinyl) phosphorothioate), an organophosphate pesticide, is widely used throughout the world. Prior to, and possibly even since, the recent agreement between the major manufacturer, Dow AgroSciences, and the EPA limiting the use of chlorpyrifos, it was the most widely used pesticide in the United States. Its primary target of toxicity is the CNS. Because of the large number of humans that are exposed to chlorpyrifos, we used chlorpyrifos as a model pesticide to investigate the role of pesticide exposure in the induction of neuronal apoptosis and to elucidate the underlying molecular mechanisms.

Chlorpyrifos is a well-known acetylcholinesterase (AChE) inhibitor. However, recent concern has focused on its neurotoxicity independent of AChE inhibition (Auman et al., 2000Go; Crumpton et al., 2000bGo). In this study, we investigated whether chlorpyrifos induces apoptosis in primary cultured CNS neurons. We chose primary cultured cortical neurons from newborn rats as our in vitro model system because the neocortex is essential to cognition as well as learning and memory. Furthermore, neurons in this region are frequently damaged during neurodegeneration.

Several signaling pathways have been implicated in the regulation of neuronal apoptosis (Hetman and Xia, 2000Go). These include the extracellular signal-regulated protein kinase (ERK) 1/2, the c-Jun NH2-terminal protein kinase (JNK), and the p38 MAP kinase. ERK1/2 are preferentially activated by growth factors and neurotrophic factors, while JNK and p38 are preferentially activated by cell stress-inducing signals, such as oxidative stress, environmental stress, and toxic chemical insults (Davis, 2000Go). Using nerve growth factor (NGF)-withdrawal induced PC12 cell apoptosis as a model, it was discovered that activation of the JNK and p38 MAP kinases induce apoptosis while the ERK1/2 protect against this apoptosis (Xia et al., 1995Go). It has been hypothesized that the balance between growth factor activated survival pathways and stress-activated death pathways determines whether a cell survives or dies (Xia et al., 1995Go). Although transient activation of JNK or p38 has been implicated in proliferation and even cellular survival, many subsequent studies have confirmed an anti-apoptotic role for ERK1/2 and a pro-apoptotic role for sustained activation of p38 and JNK (Figueroa-Masot et al., 2001Go; Hetman and Xia, 2000Go; Ip and Davis, 1998Go; Kawasaki et al., 1997Go; Namgung and Xia, 2000Go; Yang et al., 1997Go).

In the present study, an in vitro primary neuron culture system was used to assess whether exposure to chlorpyrifos or its metabolites could induce apoptosis and to elucidate the underlying signaling mechanisms. Our data indicate that chlorpyrifos and chlorpyrifos-oxon, but not TCP, induce neuronal apoptosis. Furthermore, we implicate several MAP kinase signaling pathways in chlorpyrifos-induced neurotoxicity.

MATERIALS AND METHODS

Primary cortical neuron cultures.
Cortical neurons were prepared from newborn (P0) or embryonic day 17/18 (E17/18) Sprague-Dawley rats (B&K, Kent, WA) as described (Xia et al., 1996Go) with modifications. Briefly, dissociated cortical neurons were plated at a density of 2 x 106 cells per 35 mm plate, and cultured in Neurobasal A (Gibco BRL, Rockville, MD) medium supplemented with 2% B27 (Gibco BRL), 25 mM glutamate, 18 mM glucose, 0.5 mM L-glutamine, 50 U/ml penicillin, and 0.05 mg/ml streptomycin in a humidified incubator with 5% CO2 at 37°C. Half of the culture medium was changed every 2-3 days with fresh medium excluding glutamate. Plates and glass coverslips (Bellco, Vineland, NJ) were coated with poly-D-lysine and laminin (Collaborative Biomedical Product, Bedford, MA).

Drug treatment.
Cortical neurons were cultured for 5-7 days before chlorpyrifos, chlorpyrifos-oxon, or TCP exposure. Chlorpyrifos (99.2% pure) and chlorpyrifos-oxon (96% pure; ChemService, Inc., West Chester, PA) were dissolved in ethanol and freshly prepared for each experiment. TCP (98% pure; ChemService, Inc.) was dissolved in ethanol to a stock solution of 100 mM. Aliquots of TCP were then diluted in media and pH was adjusted with 1N NaOH before treating cells. All treatments were one-time, single-dose exposures. All compounds were handled in a chemical hood. SL327 (DuPont Pharmaceuticals Company) and SB202190 (CalBiochem, La Jolla, CA) were added to the culture 1 h before chlorpyrifos exposure.

MTT metabolism assay.
This was done as described (Hetman et al., 1999Go).

Quantitation of apoptosis by nuclear morphological changes.
To visualize nuclear morphology, cells were fixed in 4% paraformaldehyde and stained with 2.5 µg/ml of the DNA dye Hoechst 33258 (bis-benzimide; Sigma, St. Louis, MO). Apoptosis was quantitated by scoring the percentage of apoptotic cells in the adherent cell population. Uniformly stained nuclei were scored as healthy, viable neurons. Condensed or fragmented nuclei were scored as apoptotic. In order to obtain unbiased counting, slides were coded and cells were scored blind without knowledge of their prior treatment.

Transient transfection of cortical neurons, detection of transfected cells, and quantitation of apoptosis in transfected cells.
Cortical neurons were transiently transfected on the third day in vitro (DIV 3) using a calcium-phosphate co-precipitation protocol as described (Hetman et al., 1999Go; Namgung and Xia, 2000Go). Neuron cultures were always cotransfected with an expression vector encoding enhanced green fluorescent protein (eGFP) as a marker for transfected cells (Hetman et al., 1999Go; Xia et al., 1995Go). Neuron cultures were treated with chlorpyrifos at one day after transfection for 24 h and cells were harvested by fixation in a solution containing 4% paraformaldehyde and 4% sucrose. Transfected cells were identified by eGFP fluorescence. Apoptosis in transfected cells was assayed by nuclei fragmentation and condensation after staining with Hoechst 33258 (2.5 µg/ml; Hetman et al., 1999Go; Xia et al., 1995Go). Transfected cells were scored blind for apoptosis under a fluorescence microscope at the single neuron level. The percentage of apoptotic cells in the total transfected cell population was quantitated.

Western analysis.
This was done as described (Figueroa-Masot et al., 2001Go). Anti-ERK2 antibody and anti-phospho specific antibodies for ERK1/2, JNK and p38 were purchased from Promega, anti-phospho c-Jun antibody from Cell Signaling Technology (Beverly, MA), anti-JNK1 from Pharmingen, anti-p38 from Santa Cruz Biotechnology (Santa Cruz, CA), and anti-ß-actin from Sigma.

Immunohistochemistry for endogenous phospho-JNK, phospho-c-Jun, and MAP-2 proteins in neurons.
This was done as described (Figueroa-Masot et al., 2001Go).

Statistical analysis.
Data are presented as means ± SE. Results are from three or more independent experiments. We employed one-way ANOVA for statistical analysis of the data. Data with statistical values of p < 0.05 are generally accepted as statistically significant (Zar, 1996Go).

RESULTS

Chlorpyrifos Disrupts Mitochondria Function and Induces Apoptosis in Postnatal Cortical Neurons
To examine the neurotoxic effects of chlorpyrifos, cortical neurons prepared from newborn rats (P0) were treated for 24, 48, and 72 h to various concentrations of chlorpyrifos or to vehicle control ethanol (0.015% final concentration; Fig. 1). The neurotoxic effect of chlorpyrifos was first measured using the MTT metabolism assay (Hetman et al., 1999Go). The MTT assay is based on the colorimetric conversion of the yellow, water-soluble tetrazolium, MTT (3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide) to the blue, water-insoluble formazan (Mosmann, 1983Go). This conversion is catalyzed by cellular mitochondrial dehydrogenases. Because the rate of this reaction is proportional to the number of surviving cells and functional mitochondria, the MTT assay is widely used to quantify viable cells and to assess mitochondrial function (Mosmann, 1983Go). Chlorpyrifos reduced MTT metabolism in a dose- and time-dependent manner (Fig. 1), suggesting that chlorpyrifos decreases cortical neuron viability and/or interferes with mitochondrial function.



View larger version (27K):
[in this window]
[in a new window]
 
FIG. 1. Chlorpyrifos decreases MTT metabolism in cortical neurons. Cortical neurons were prepared from newborn (P0) rats. Five days after culture in vitro (DIV 5), cells were treated with various concentrations of chlorpyrifos (0-80 µM) for 0, 24, 48, and 72 h and MTT metabolism measured.

 
To determine if chlorpyrifos induces apoptosis in cortical neurons, neurons were stained with the DNA dye Hoechst 33258 to visualize nuclear morphology (Fig. 2). In the absence of chlorpyrifos, the cultured neurons exhibited evenly stained nuclei (Fig. 2A). Chlorpyrifos caused morphological changes characteristic of apoptosis, including nuclei fragmentation and condensation (Fig. 2B). The percentage of neurons showing an apoptotic phenotype was quantitated as a function of the concentration and time after chlorpyrifos exposure (Fig. 2C). Treatment with 50 µM chlorpyrifos (equal to 17.5 µg/ml) increased apoptosis from a 30% basal level to 59% at 24 h, and 76% at 72 h.



View larger version (22K):
[in this window]
[in a new window]
 
FIG. 2. Chlorpyrifos induces apoptosis in cortical neurons. P0 (A-C) and E17 (D) cortical neurons at DIV 5 were treated with 0, 30, or 50 µM chlorpyrifos for 0, 24, 48, and 72 h. (A and B) Representative Hoechst nuclei staining photomicrographs of vehicle control (A) or chlorpyrifos (B) treated cortical neurons. Arrows in panel A demonstrate healthy nuclei, while arrows in panel B indicate apoptotic nuclei that are fragmented and/or condensed. (C and D) Quantitation of apoptosis in P0 (C) or E17 (D) neurons. Results are averages of three independent experiments done in triplicate. Error bars are standard error of the mean (SE). **p < 0.005.

 
The increase in nuclear fragmentation and condensation generally precedes the reduction in MTT metabolism in apoptotic cells because early apoptotic cells maintain mitochondria function and MTT metabolism (Hetman et al., 1999Go; Namgung and Xia, 2000Go; Thompson, 1995Go). However, in chlorpyrifos-treated neurons, MTT metabolism was severely impaired prior to nuclear morphological changes. For example, while 30 µM chlorpyrifos did not cause much change in the levels of apoptosis measured by nuclear fragmentation and condensation up to 72 h after the exposure (Fig. 2C), it caused a 40 and 55% reduction in MTT metabolism at 24 and 72 h, respectively (Fig. 1). Treatment with 15 µM chlorpyrifos (equal to 5.25 µg/ml) also caused a reproducible level of reduction in MTT metabolism. These data suggest that the reduction in MTT metabolism is not a result of cells undergoing late-stage apoptosis. Rather, chlorpyrifos disrupts mitochondrial function of cortical neurons even before it caused apoptosis.

Chlorpyrifos-Oxon Induces Apoptosis in Postnatal Cortical Neurons
Chlorpyrifos-oxon is the active metabolite of chlorpyrifos. Because many of the toxic effects of chlorpyrifos have been attributed to chlorpyrifos-oxon inhibition of the brain AChE activity (Ecobichon, 2001Go), we examined the effect of chlorpyrifos-oxon on P0 cortical neuron apoptosis. P0 cortical neurons were treated with 0 (vehicle control), 20, or 50 µM chlorpyrifos-oxon, and its effect on MTT metabolism or apoptosis examined at 24, 48, and 72 h following exposure. Like chlorpyrifos, chlorpyrifos-oxon reduced MTT metabolism (Fig. 3A) and induced apoptosis (Fig. 3B) in cortical neurons. The dose response and kinetics of MTT metabolism were very similar in neurons treated with chlorpyrifos and chlorpyrifos-oxon. At 50 µM concentrations, chlorpyrifos-oxon did not induce more apoptosis than chlorpyrifos. Although 30 µM chlorpyrifos did not induce an appreciable level of apoptosis, 20 µM chlorpyrifos-oxon was able to induce apoptosis in these neurons from basal level of 29 to 51% at 72 h post treatment. This suggests that at lower concentrations, these neurons are more sensitive to chlorpyrifos-oxon than to the parent compound. Chlorpyrifos-oxon is about three orders of magnitude more potent than chlorpyrifos in inhibition of brain AChE activity (Das and Barone, 1999Go; Huff et al., 1994Go). Thus, if chlorpyrifos-induced apoptosis is mediated by AChE inhibition, the oxon form should induce apoptosis at concentrations two to three orders of magnitude lower than chlorpyrifos. Because chlorpyrifos-oxon was only slightly more potent than chlorpyrifos in inducing apoptosis, our data suggest that chlorpyrifos-induced apoptosis is unlikely due to AChE inhibition.



View larger version (16K):
[in this window]
[in a new window]
 
FIG. 3. Chlorpyrifos-oxon induces apoptosis in cortical neurons. P0 cortical neurons at DIV 5 were treated with 0, 20, or 50 µM of chlorpyrifos-oxon for 0, 24, 48, and 72 h. (A) Dose response and kinetics of MTT metabolism. (B) Quantitation of apoptosis. Results are averages of three independent experiments done in triplicate. Error bars: SE.

 
TCP Does Not Induce Apoptosis in Postnatal Cortical Neurons
TCP (3,5,6-trichloro-2-pyridinol) is the major breakdown product of the detoxification of chlorpyrifos and chlorpyrifos-oxon. It is generally thought to be nontoxic because it has no effect on AChE. However, it has been suggested that TCP may cross the placental barrier and accumulate in the fetal brain (Hunter et al., 1999Go). P0 cortical neurons were treated with various concentrations of TCP for 24, 48, and 72 h. After 72 h exposure, TCP caused a 34% reduction in MTT metabolism at 150 µM (Fig. 4A). However, it did not have any effect at 50 µM, at which concentration chlorpyrifos or chlorpyrifos-oxon inhibited MTT metabolism by 75-80%. TCP did not induce any apoptosis even after 72 h treatment at 130 µM (Fig. 4B). These results suggest that TCP causes minimal toxicity in cultured cortical neurons compared to the parent compounds.



View larger version (17K):
[in this window]
[in a new window]
 
FIG. 4. TCP does not induce apoptosis in cortical neurons. P0 cortical neurons at DIV 5 were treated with various concentrations of TCP for 0, 24, 48, and 72 h. (A) Dose response and kinetics of MTT metabolism. (B) Quantitation of apoptosis. Results are averages of three independent experiments done in triplicate. Error bars: SE.

 
Embryonic Neurons Are More Sensitive to Chlorpyrifos Than Postnatal Neurons
During different stages of neurodevelopment neurons can be more or less sensitive to the same stimuli. Therefore, we assessed whether embryonic neurons would display a different sensitivity to chlorpyrifos. Cortical neurons were prepared from E17 rats, and treated with 0 (vehicle control), 30, or 50 µM chlorpyrifos. Apoptosis was examined at 24, 48, and 72 h (Fig. 2D). Although 30 µM chlorpyrifos did not induce apoptosis in P0 cortical neurons, it caused a statistically significant increase (p < 0.005) in apoptosis in E17 cortical neurons. Furthermore, 50 µM chlorpyrifos increased apoptosis from 21% basal level to 92% after 72 h treatment. However, E17 neurons were not more sensitive to chlorpyrifos-oxon or TCP than P0 neurons (data not shown). These data suggest that embryonic neurons are more sensitive to chlorpyrifos-induced apoptosis than postnatal neurons.

ERK1/2 is Pro-apoptotic in Chlorpyrifos-Induced Cortical Neuron Apoptosis
To determine if various MAP kinase signaling pathways are involved in the regulation of chlorpyrifos-induced apoptosis, we first determined whether they are activated by chlorpyrifos. ERK1/2 activation was assayed by Western analysis using an anti-phospho-ERK1/2 antibody that specifically recognizes activated ERK1/2. Chlorpyrifos (50 µM) induced ERK1/2 phosphorylation (Fig. 5A), indicative of its activation. This activation persisted for 4 h. The importance of ERK1/2 activation for induction of apoptosis was investigated by inhibition of ERK1/2 using SL327. SL327 is a water soluble, structural homologue of the specific MKK1/2 inhibitor U0126 and has been widely used to study neuronal signaling mechanisms (Atkins et al., 1998Go; Hetman et al., 2002Go). MKK1/2 are upstream kinases that phosphorylate and activate ERK1/2 (Lewis et al., 1998Go). Treatment with 50 µM SL327 caused a partial but statistically significant (p < 0.005) reduction in chlorpyrifos-induced apoptosis (Fig. 5B). These data suggest that activation of the ERK1/2 signaling pathway contributes to the induction of cortical neuron apoptosis after chlorpyrifos treatment.



View larger version (21K):
[in this window]
[in a new window]
 
FIG. 5. Role of ERK1/2 in chlorpyrifos-induced apoptosis in cortical neurons. (A) Chlorpyrifos activates ERK1/2 in cortical neurons. P0 cortical neurons at DIV 5 were treated with 50 µM chlorpyrifos for the indicated times and 20 µg of total protein was subjected to western analysis. ERK1/2 activity was determined by western analysis using an antibody recognizing phosphorylated and activated (p-) ERK1/2 (top). Anti-total ERK2 western was used to normalize protein loading (bottom). Results are representatives of two independent experiments. (B) The MEK inhibitor, SL327, partially protects neurons from chlorpyrifos-induced apoptosis. P0 cortical neurons at DIV 5 were pretreated for 1 h with 50 µM SL327 or vehicle control, then exposed to 50 µM chlorpyrifos (CPF) or vehicle control. Apoptosis was scored 24 h later. Results are average of three independent experiments ± SE. **p < 0.005.

 
The p38 Signaling Pathway is Activated following Chlorpyrifos Exposure and is a Cell-Intrinsic Survival Pathway to Counteract Stress
To evaluate the contribution of p38 for chlorpyrifos-induced apoptosis in cortical neurons, we determined if p38 is activated in response to chlorpyrifos treatment. Activation of p38 was measured by Western analysis using an anti-phospho-p38 antibody that specifically recognizes phosphorylated and activated p38 (Fig. 6A). Activation of p38 was evident within 0.5 h of chlorpyrifos treatment; this activation persisted for 24 h. The importance of p38 activation for induction of apoptosis was investigated by inhibition of p38. Treatment with 10 µM SB202190, a specific inhibitor for p38, greatly accelerated chlorpyrifos-induced cortical neuron apoptosis (Fig. 6B). These results suggest that p38 is activated as a survival signaling pathway in response to chlorpyrifos treatment to counteract the apoptotic signal.



View larger version (22K):
[in this window]
[in a new window]
 
FIG. 6. p38 MAP kinase is activated as a compensatory survival response in chlorpyrifos-treated cortical neurons. (A) Chlorpyrifos activates p38 in cortical neurons. P0 cortical neurons at DIV 5 were treated with 50 µM chlorpyrifos for the indicated times and 20 µg of total protein were subjected to Western analysis. p38 activity was determined by Western analysis using an antibody recognizing phosphorylated and activated (p-) p38 (top). Anti-total p38 Western (bottom) was used to normalize protein loading. Results are representatives of two independent experiments. (B) The p38 inhibitor, SB202190, greatly potentiates chlorpyrifos-induced apoptosis. P0 cortical neurons at DIV 5 were pretreated for 1 h with 10 µM SB202190 or vehicle control, then exposed to 50 µM chlorpyrifos (CPF) or vehicle control. Apoptosis was scored 24 h later. Results are average of three independent experiments ± SE. ***p < 0.001.

 
Chlorpyrifos Activates a Sub-pool of JNK in the Nucleus of Cortical Neurons
To determine if JNK contributes to chlorpyrifos-induced apoptosis in cortical neurons, JNK activity was assayed after treatment with 50 µM CPF for various times in P0 cortical neurons. Total JNK activity in whole cell lysates was assayed by Western analysis using an antibody that recognizes phosphorylated and activated JNK (anti-p-JNK). Data in Fig. 7A show that chlorpyrifos did not cause any increase in total phospho-JNK, suggesting that there was no increase in total JNK activity. Interestingly, there was a robust increase in phosphorylation of c-Jun (Fig. 7B), a nuclear transcription factor and downstream target of JNK. c-Jun phosphorylation was detectable at 4 h and persisted until at least 24 h after treatment. This suggests the interesting possibility that although total JNK is not activated, a sub-pool of JNK, e.g., the nuclear pool of JNK, may be activated by chlorpyrifos. Indeed, immunocytochemistry studies using phospho-JNK and phospho-c-Jun antibodies demonstrated that basal p-JNK in cortical neurons was primarily present in the cell bodies and processes, but absent in the nucleus (Fig. 8). Chlorpyrifos caused an increase in p-JNK in the nucleus of neurons and a decrease in p-JNK in the processes. This was accompanied by an increase in c-Jun phosphorylation in the nucleus, indicative of c-Jun activation. The kinetics of c-Jun phosphorylation in the nucleus correlates with that of JNK phosphoryaltion in the nucleus, both detectable at 4 h and peaked at 8 h after chlorpyrifos treatment. Phosphorylation of JNK and c-Jun in the nucleus was as robust at 24 h as at 8 h (data not shown). These data suggest that either a sub-pool of JNK is activated in the nucleus or a sub-pool of active JNK is translocated to the nucleus in response to chlorpyrifos. Regardless, the net result is increased JNK activity in the nucleus, c-Jun phosphorylation, and presumably c-Jun stimulated, AP-1-mediated transcription.



View larger version (26K):
[in this window]
[in a new window]
 
FIG. 7. Chlorpyrifos does not activate total JNK but induces c-Jun phosphorylation. P0 cortical neurons at DIV 5 were treated with 50 µM chlorpyrifos for the indicated times and 20 µg of total protein were subjected to Western analysis using antibodies recognizing either phosphorylated and activated (p-) JNK (A) or phosphorylated and activated (p-) c-Jun (B). Antibodies to total JNK1 or ß-actin were used to verify protein loading. Results are representatives of two independent experiments.

 


View larger version (125K):
[in this window]
[in a new window]
 
FIG. 8. Chlorpyrifos stimulates phosphorylation of JNK and c-Jun in the nuclei of cortical neurons. P0 cortical neurons were treated with chlorpyrifos for 0, 4, 6, and 8 h and co-immunostained with antibodies for MAP-2, a neuron marker (a-d) and p-JNK (e-h), or MAP-2 (i-l), and p-c-Jun (m-p). MAP-2 was used to identify cell bodies and neurites of neurons. (a and e) Arrows in these panels point to the nuclei of a neuron that is devoid of p-JNK staining. (d, h, l, and p) Arrows in these panels point to neurons with nuclear staining of p-JNK and p-c-Jun. All images were captured under the same exposure conditions using a Leica confocal microscope.

 
Chlorpyrifos-Induced Apoptosis Requires JNK Activation
To elucidate a functional role of JNK and c-Jun activation in the nucleus in chlorpyrifos-induced apoptosis, we treated cortical neurons with SP600125, a JNK inhibitor (Calbiochem). However, this drug was not very effective in inhibiting JNK in our experiments (data not shown). We then transiently transfected cortical neurons with a dominant negative c-Jun and scored apoptosis in transfected cells after chlorpyrifos treatment. Transfected cells were identified by the green fluorescence of the co-transfected marker protein eGFP. Expression of the dominant negative c-Jun inhibited chlorpyrifos-induced cortical neuron apoptosis (Fig. 9). These data suggest a critical role for nuclear JNK activation and its downstream transcriptional events in chlorpyrifos-induced apoptosis in cortical neurons.



View larger version (17K):
[in this window]
[in a new window]
 
FIG. 9. Expression of a dominant negative c-Jun blocks chlorpyrifos-induced apoptosis in cortical neurons. PO cortical neurons were transfected on DIV 3 with 3 µg of plasmid DNA encoding a dominant negative (DN) c-Jun (Tam 67; Rapp et al., 1994Go) or a vector control pcDNA3. The cells were also cotransfected with 1 µg of eGFP DNA as a transfection marker. At 24 h post transfection, neurons were treated for 24 h with 80 µM chlorpyrifos or vehicle control ethanol. Apoptosis in transfected cells (GFP positive) was then quantified. Results are average of three independent experiments ± SE. **p < 0.01.

 
DISCUSSION

The objectives of this study were to determine if chlorpyrifos induces apoptosis in primary cultured CNS neurons and to elucidate the underlying mechanisms of chlorpyrifos neurotoxicity. Our data indicate that chlorpyrifos and chlorpyrifos-oxon, but not TCP, induce apoptosis in primary cortical neurons. They also reduced MTT metabolism. Furthermore, chlorpyrifos activated ERK1/2, p38, and JNK signaling pathways. Blocking ERK1/2 or JNK signaling inhibited chlorpyrifos-induced apoptosis, suggesting that ERK1/2 and JNK act as pro-apoptotic pathways to mediate chlorpyrifos-induced apoptosis. In contrast, blocking p38 signaling potentiated chlorpyrifos-induced apoptosis, indicating that p38 is activated as a compensatory survival mechanism to counteract chlorpyrifos neurotoxicity.

Much of chlorpyrifos neurotoxicity has been attributed to AChE inhibition via its active metabolite chlorpyrifos-oxon. However, recent studies suggest that additional mechanisms may also be at play. For example, there is evidence to suggest that the effects of chlorpyrifos on the developing brain could be quantitatively and/or qualitatively different from that on adult brain (Atterberry et al., 1997Go; Won et al., 2001Go). Immature animals, although able to recover from AChE inhibition much more rapidly than their adult counterparts, are more sensitive to chlorpyrifos than to chlorpyrifos-oxon (Pope et al., 1991Go; Song et al., 1997Go; Whitney et al., 1995Go). Consequently, recent research has focused on identifying chlorpyrifos toxicity independent of its effect on AChE inhibition (Auman et al., 2000Go; Crumpton et al., 2000bGo). For example, chlorpyrifos inhibits macromolecule and protein synthesis in PC12 cells that cannot be blocked by muscarinic or nicotinic antagonists (Song et al., 1998Go). Our data show that both chlorpyrifos and chlorpyrifos-oxon induce apoptosis in cortical neurons. These results are consistent with a previous report that chlorpyrifos causes mitotic abnormalities and apoptosis in neuroepithelium of cultured rat embryos (Roy et al., 1998Go). Chlorpyrifos-oxon is about two to three orders of magnitude more potent than chlorpyrifos in inhibition of brain AChE activity. However, chlorpyrifos-oxon was only slightly more potent than chlorpyrifos in inducing cortical neuron apoptosis. Thus chlorpyrifos-induced apoptosis in primary cortical neurons may occur independently of AChE, although AChE activity was not measured in this study. It will be interesting to investigate if there is any correlation between AChE inhibition and chlorpyrifos-induced apoptosis.

TCP is the major breakdown product of the detoxification of chlorpyrifos and chlorpyrifos-oxon. It is generally thought to be nontoxic. Our results showed that TCP does not induce cortical neuron apoptosis at concentrations up to 130 µM, consistent with reports in the literature that TCP is a relatively nontoxic metabolite. However, because TCP may selectively accumulate in fetal brain (Hunter et al., 1999Go), our data do not exclude a role for TCP in mediating the developmental neurotoxicity of chlorpyrifos.

Interestingly, chlorpyrifos treatment decreased MTT metabolism before it caused nuclear morphological changes characteristic of apoptosis. Many apoptotic insults cause changes in nuclear morphology before they have a detectable effect on MTT metabolism because early apoptotic cells maintain mitochondria function and MTT metabolism (Hetman et al., 1999Go; Namgung and Xia, 2000Go; Thompson, 1995Go). However, for some other insults, disruption of mitochondrial potential is an early event and precedes nuclear changes (Dumont et al., 1999Go; Kroemer et al., 1997Go; Stridh et al., 1998Go; Zamzami et al., 1996Go). Our data suggest that mitochondrial inhibition resulting from chlorpyrifos may be an early biomarker of pesticide exposure, perhaps shifting the balance towards apoptosis.

Chlorpyrifos reduced MTT metabolism in cortical neurons at low concentrations that did not induce apoptosis assayed by nuclear fragmentation and condensation. For example, although low concentrations of chlorpyrifos (< 30 µM) reduced MTT metabolism as early as 24 h, they did not induce apoptosis even after 72 h treatment. Therefore, chlorpyrifos may interfere with mitochondrial function at low concentrations without causing apoptosis, and MTT metabolism may be an early biomarker for chlorpyrifos toxicity. Although MTT metabolism is generally used to assay for cell viability, our data open the possibility that MTT metabolism may not always correlate with cell viability because a reduction in MTT metabolism may occur without an ultimate decrease in cell viability. For instance, mitochondrial function may be repaired without losing cell viability. This is exemplified in glutamate treated cells in which neurons surviving the early necrotic phase recover from mitochondrial potential and energy levels (Ankarcrona et al., 1995Go). Alternatively, chlorpyrifos may induce an early phase of necrosis, thus the decrease in MTT metabolism in a subpopulation of the neurons. Those neurons surviving the necrotic phase may die by apoptosis at higher concentration. This latter case is less likely because we did not observe morphological changes associated with necrosis when cells were treated with chlorpyrifos at concentrations lower than 30 µM. Regardless, apoptosis induced by chlorpyrifos treatment is one of the major and novel manifestations of chlorpyrifos toxicity.

There are few or no data regarding the relative distribution or concentration of chlorpyrifos in human brain after exposure. In the literature, 50-150 µM chlorpyrifos has been routinely used to study chlorpyrifos toxicity (Crumpton et al., 2000aGo; Dam et al., 1999Go; Jett and Navoa, 2000Go; Roy et al., 1998Go). In this study, we used 30-80 µM (10.5-28 µg/ml) chlorpyrifos, concentrations consistent with most of the previous studies examining chlorpyrifos toxicity. Interestingly, co-exposure of chlorpyrifos with rotenone synergistically induced neuronal apoptosis in a neuronal cell line (Wang and Xia, unpublished observation), suggesting that chlorpyrifos may synergize with other pesticides to induce neuronal apoptosis.

Our data showed that chlorpyrifos and its oxon metabolite induce apoptosis in cortical neurons. Because chlorpyrifos is a widely used pesticide and apoptosis plays an important role in the pathology of many forms of neurodegeneration, our results support the hypothesis that pesticide exposure may increase the risk of neurodegeneration. Because the brains of newborn rats are developmentally equivalent to the human brain in the third trimester, and E17 neurons are even more sensitive than P0 neurons to chlorpyrifos-induced apoptosis, fetuses whose mothers are slow metabolizers of chlorpyrifos may be at an increased risk of toxicity from chlorpyrifos-induced apoptosis. Furthermore, changes in the amount of apoptosis in the developing brain could cause structural changes that in turn could lead to developmental deficits in many areas of the brain function, including learning and memory. Thus our data may have implications on early developmental exposure of pesticides on children's learning and memory.

It is interesting that chlorpyrifos exposure caused sustained activation of three families of MAP kinases, the ERK1/2, the p38, and the JNK. Activation of ERK1/2 usually protects neurons from apoptosis, while sustained activation of p38 and JNK by stress stimuli often causes neuron cell death (Figueroa-Masot et al., 2001Go; Hetman et al., 1999Go; Hetman and Xia, 2000Go; Ip and Davis, 1998Go; Kawasaki et al., 1997Go; Namgung and Xia, 2000Go; Xia et al., 1995Go; Yang et al., 1997Go). Surprisingly, here we report that blocking ERK1/2 activation protected neurons from, while blocking p38 accelerated apoptosis induced by chlorpyrifos. These data suggest that ERK1/2 contribute to chlorpyrifos-induced apoptosis. Furthermore, to our knowledge, this is the first evidence that p38 is activated by an insult as a compensatory mechanism.

Although chlorpyrifos did not induce JNK phosphorylation when total cell lysates were used for Western analysis, it increased nuclear JNK phosphorylation, indicative of nuclear JNK activation. Moreover, chlorpyrifos stimulated c-Jun phosphorylation, providing additional evidence for nuclear JNK activation. Expression of a dominant negative c-Jun inhibited chlorpyrifos-induced apoptosis. These data suggest that nuclear JNK activation and JNK stimulated, c-Jun-mediated transcription play a critical role in chlorpyrifos-induced cortical neuron apoptosis. The fact that a distinct pool of JNK is activated in the nucleus by chlorpyrifos is reminiscent to nuclear JNK activation by taxol (Figueroa-Masot et al., 2001Go). These results suggest that specific nuclear JNK activation may be a general mechanism of insult-induced apoptosis in CNS neurons.

In summary, we discovered that chlorpyrifos induces apoptosis and reduces mitochondrial function of cortical neurons. This apoptosis may be independent of AChE inhibition and is regulated by a complex of MAP kinases including ERK1/2, JNK, and p38. Activation of these MAP kinases may also underlie other aspects of chlorpyrifos neurotoxicity in addition to apoptosis. Taken together, our results indicate apoptosis and perturbation of mitochondrial function as novel toxic endpoints of chlorpyrifos-induced neurotoxicity, which may have implications for pesticide exposure in humans both during development and in neurodegeneration in adults.

ACKNOWLEDGMENTS

We thank DuPont Pharmaceuticals Company for providing us with SL327. This work was supported by APP#3010 from Burroughs Wellcome Fund for New Investigator Award in Toxicology (Z.X.), the UW NIEHS sponsored Center for Ecogenetics and Environmental Health (NIEHS P30ES07033), grants NS44069 and EPA-R826886/ES09601.

NOTES

1 These authors contributed equally to this work. Back

2 Present address: Department of Oriental Medicine, College of Oriental Medicine, Daejeon University, South Korea. Back

REFERENCES

Ankarcrona, M., Dypbukt, J. M., Bonfoco, E., Zhivotovsky, B., Orrenius, S., Lipton, S. A., and Nicotera, P. (1995). Glutamate-induced neuronal death: A succession of necrosis or apoptosis depending on mitochondrial function. Neuron 15, 961–973.[ISI][Medline]

Atkins, C. M., Selcher, J. C., Petraitis, J. J., Trzaskos, J. M., and Sweatt, J. D. (1998). The MAPK cascade is required for mammalian associative learning. Nature Neurosci. 1, 602–609.[CrossRef][ISI][Medline]

Atterberry, T. T., Burnett, W. T., and Chambers, J. E. (1997). Age-related differences in parathion and chlorpyrifos toxicity in male rats: Target and nontarget esterase sensitivity and cytochrome P450-mediated metabolism. Toxicol. Appl. Pharmacol. 147, 411–418.[CrossRef][ISI][Medline]

Auman, J. T., Seidler, F. J., and Slotkin, T. A. (2000). Neonatal chlorpyrifos exposure targets multiple proteins governing the hepatic adenylyl cyclase signaling cascade: Implications for neurotoxicity. Brain Res. Dev. Brain Res. 121, 19–27.[ISI][Medline]

Checkoway, H., and Nelson, L. M. (1999). Epidemiologic approaches to the study of Parkinson's disease etiology. Epidemiology 10, 327–336.[ISI][Medline]

Crumpton, T. L., Seidler, F. J., and Slotkin, T. A. (2000a). Developmental neurotoxicity of chlorpyrifos in vivo and in vitro: Effects on nuclear transcription factors involved in cell replication and differentiation. Brain Res. 857, 87–98.[CrossRef][ISI][Medline]

Crumpton, T. L., Seidler, F. J., and Slotkin, T. A. (2000b). Is oxidative stress involved in the developmental neurotoxicity of chlorpyrifos? Brain Res. Dev. Brain Res. 121, 189–195.[ISI][Medline]

Dam, K., Seidler, F. J., and Slotkin, T. A. (1999). Chlorpyrifos releases norepinephrine from adult and neonatal rat brain synaptosomes. Brain Res. Dev. Brain Res. 118, 129–133.[ISI][Medline]

Das, K. P., and Barone, S., Jr. (1999). Neuronal differentiation in PC12 cells is inhibited by chlorpyrifos and its metabolites: Is acetylcholinesterase inhibition the site of action? Toxicol. Appl. Pharmacol. 160, 217–230.[CrossRef][ISI][Medline]

Davis, R. J. (2000). Signal transduction by the JNK group of MAP kinases. Cell 103, 239–252.[ISI][Medline]

Di Monte, D. A., Lavasani, M., and Manning-Bog, A. B. (2002). Environmental factors in Parkinson's disease. Neurotoxicology 23, 487–502.[CrossRef][ISI][Medline]

Dumont, A., Hehner, S. P., Hofmann, T. G., Ueffing, M., Droge, W., and Schmitz, M. L. (1999). Hydrogen peroxide-induced apoptosis is CD95-independent, requires the release of mitochondria-derived reactive oxygen species and the activation of NF-kappaB. Oncogene 18, 747–757.[CrossRef][ISI][Medline]

Ecobichon, D. J. (2001). Toxic effects of pesticides. In Casarett & Doull's Toxicology: The Basic Science of Poisons. 6th edition (C. D. Klaassen, Ed.), pp. 763–810. McGraw-Hill, New York.

Figueroa-Masot, X. A., Hetman, M., Higgins, M. J., Kokot, N., and Xia, Z. (2001). Taxol induces apoptosis in cortical neurons by a mechanism independent of Bcl-2 phosphorylation. J. Neurosci. 21, 4657–4667.[Abstract/Free Full Text]

Guillette, E. A., Meza, M. M., Aquilar, M. G., Soto, A. D., and Garcia, I. E. (1998). An anthropological approach to the evaluation of preschool children exposed to pesticides in Mexico. Environ. Health Perspect. 106, 347–353.[ISI][Medline]

Hetman, M., Hsuan, S. L., Habas, A., Higgins, M. J., and Xia, Z. (2002). ERK1/2 antagonizes glycogen synthase kinase-3beta -induced apoptosis in cortical neurons. J. Biol. Chem. 277, 49577–49584.[Abstract/Free Full Text]

Hetman, M., Kanning, K., Cavanaugh, J. E., and Xia, Z. (1999). Neuroprotection by brain-derived neurotrophic factor is mediated by extracellular-signal-regulated kinase and phosphatidylinositol-3 kinase. J. Biol. Chem. 274, 22569–22580.[Abstract/Free Full Text]

Hetman, M., and Xia, Z. (2000). Signaling pathways mediating anti-apoptotic action of neurotrophins. Acta Neurobiol. Exp. (Warsz) 60, 531–545.[ISI][Medline]

Huff, R. A., Corcoran, J. J., Anderson, J. K., and Abou-Donia, M. B. (1994). Chlorpyrifos oxon binds directly to muscarinic receptors and inhibits cAMP accumulation in rat striatum. J. Pharmacol. Exp. Ther. 269, 329–335.[Abstract]

Hunter, D. L., Lassiter, T. L., and Padilla, S. (1999). Gestational exposure to chlorpyrifos: Comparative distribution of trichloropyridinol in the fetus and dam. Toxicol. Appl. Pharmacol. 158, 16–23.[CrossRef][ISI][Medline]

Ip, Y. T., and Davis, R. J. (1998). Signal transduction by the c-Jun N-terminal kinase (JNK) - from inflammation to development. Curr. Opin. Cell Biol. 10, 205–219.[CrossRef][ISI][Medline]

Jett, D. A., and Navoa, R. V. (2000). In vitro and in vivo effects of chlorpyrifos on glutathione peroxidase and catalase in developing rat brain. Neurotoxicology 21, 141–145.[ISI][Medline]

Kawasaki, H., Morooka, T., Shimohama, S., Kimura, J., Hirano, T., Gotoh, Y., and Nishida, E. (1997). Activation and involvement of p38 mitogen-activated protein kinase in glutamate-induced apoptosis in rat cerebellar granule cells. J. Biol. Chem. 272, 18518–18521.[Abstract/Free Full Text]

Kroemer, G., Zamzami, N., and Susin, S. A. (1997). Mitochondrial control of apoptosis. Immunol. Today 18, 44–51.

Le Couteur, D. G., McLean, A. J., Taylor, M. C., Woodham, B. L., and Board, P. G. (1999). Pesticides and Parkinson's disease. Biomed. Pharmacother. 53, 122–130.[CrossRef][ISI][Medline]

Levin, E. D., Addy, N., Baruah, A., Elias, A., Christopher, N. C., Seidler, F. J., and Slotkin, T. A. (2002). Prenatal chlorpyrifos exposure in rats causes persistent behavioral alterations. Neurotoxicol. Teratol. 24, 733–741.[CrossRef][ISI][Medline]

Levin, E. D., Addy, N., Nakajima, A., Christopher, N. C., Seidler, F. J., and Slotkin, T. A. (2001). Persistent behavioral consequences of neonatal chlorpyrifos exposure in rats. Brain Res. Dev. Brain Res 130, 83–89.[ISI][Medline]

Lewis, T. S., Shapiro, P. S., and Ahn, N. G. (1998). Signal transduction through MAP kinase cascades. Adv. Cancer Res. 74, 49–139.[ISI][Medline]

Mosmann, T. (1983). Rapid colorimetric assay for cellular growth and survival: Application to proliferation and cytotoxicity assays. J. Immunol. Methods 65, 55–63.[CrossRef][ISI][Medline]

Namgung, U., and Xia, Z. (2000). Arsenite-induced apoptosis in cortical neurons is mediated by c-Jun N-terminal protein kinase 3 and p38 mitogen-activated protein kinase. J. Neurosci. 20, 6442–6451.[Abstract/Free Full Text]

Nelson, L. M. (1995). Epidemiology of ALS. Clin. Neurosci. 3, 327–331.[Medline]

Pope, C. N., Chakraborti, T. K., Chapman, M. L., Farrar, J. D., and Arthun, D. (1991). Comparison of in vivo cholinesterase inhibition in neonatal and adult rats by three organophosphorothioate insecticides. Toxicology 68, 51–61.[CrossRef][ISI][Medline]

Rapp, U. R., Troppmair, J., Beck, T., and Birrer, M. J. (1994). Transformation by raf and other oncogenes renders cells differentially sensitive to growth inhibition by a dominant negative c-jun mutant. Oncogene 9, 3493–3498.[ISI][Medline]

Roy, T. S., Andrews, J. E., Seidler, F. J., and Slotkin, T. A. (1998). Chlorpyrifos elicits mitotic abnormalities and apoptosis in neuroepithelium of cultured rat embryos. Teratology 58, 62–68.[CrossRef][ISI][Medline]

Song, X., Seidler, F. J., Saleh, J. L., Zhang, J., Padilla, S., and Slotkin, T. A. (1997). Cellular mechanisms for developmental toxicity of chlorpyrifos: Targeting the adenylyl cyclase signaling cascade. Toxicol. Appl. Pharmacol. 145, 158–174.[CrossRef][ISI][Medline]

Song, X., Violin, J. D., Seidler, F. J., and Slotkin, T. A. (1998). Modeling the developmental neurotoxicity of chlorpyrifos in vitro: Macromolecule synthesis in PC12 cells. Toxicol. Appl. Pharmacol. 151, 182–191.[CrossRef][ISI][Medline]

Stridh, H., Kimland, M., Jones, D. P., Orrenius, S., and Hampton, M. B. (1998). Cytochrome c release and caspase activation in hydrogen peroxide- and tributyltin-induced apoptosis. FEBS Lett. 429, 351–355.[CrossRef][ISI][Medline]

Terry, A. V., Jr., Stone, J. D., Buccafusco, J. J., Sickles, D. W., Sood, A., and Prendergast, M. A. (2003). Repeated exposures to subthreshold doses of chlorpyrifos in rats: Hippocampal damage, impaired axonal transport, and deficits in spatial learning. J. Pharmacol. Exp. Ther. 305, 375–384.[Abstract/Free Full Text]

Thompson, C. B. (1995). Apoptosis in the pathogenesis and treatment of disease. Science 267, 1456–1462.[ISI][Medline]

Whitney, K. D., Seidler, F. J., and Slotkin, T. A. (1995). Developmental neurotoxicity of chlorpyrifos: Cellular mechanisms. Toxicol. Appl. Pharmacol. 134, 53–62.[CrossRef][ISI][Medline]

Won, Y. K., Liu, J., Olivier, K., Jr., Zheng, Q., and Pope, C. N. (2001). Age-related effects of chlorpyrifos on acetylcholine release in rat brain. Neurotoxicology 22, 39–48.[CrossRef][ISI][Medline]

Xia, Z., Dickens, M., Raingeaud, J., Davis, R. J., and Greenberg, M. E. (1995). Opposing effects of ERK and JNK-p38 MAP kinases on apoptosis. Science 270, 1326–1331.[Abstract]

Xia, Z., Dudek, H., Miranti, C. K., and Greenberg, M. E. (1996). Calcium influx via the NMDA receptor induces immediate early gene transcription by a MAP kinase/ERK-dependent mechanism. J. Neurosci. 16, 5425–5436.[Abstract/Free Full Text]

Yang, D. D., Kuan, C. Y., Whitmarsh, A. J., Rincon, M., Zheng, T. S., Davis, R. J., Rakic, P., and Flavell, R. A. (1997). Absence of excitotoxicity-induced apoptosis in the hippocampus of mice lacking the Jnk3 gene. Nature 389, 865–870.[CrossRef][ISI][Medline]

Zamzami, N., Susin, S. A., Marchetti, P., Hirsch, T., Gomez-Monterrey, I., Castedo, M., and Kroemer, G. (1996). Mitochondrial control of nuclear apoptosis. J. Exp. Med. 183, 1533–1544.[Abstract]

Zar, J. H. (1996). Multiple comparisons. In Biostatistical Analysis (J. H. Zar, Ed.), pp. 151–161. Prentis, Hall, NJ.