A Robust Structure-Activity Relationship (SAR) Model for Esters that Cause Skin Irritation in Humans

Jeffrey S. Smith*, Orest T. Macina*, Nancy B. Sussman*, Michael I. Luster{dagger} and Meryl H. Karol*,1

* Department of Environmental and Occupational Health, University of Pittsburgh, Pittsburgh, Pennsylvania 15238; and {dagger} NIOSH, Toxicology and Molecular Biology Branch, Morgantown, West Virginia 26505

Received June 3, 1999; accepted December 20, 1999


    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
A structure-activity relationship (SAR) model has been developed to discriminate skin irritant from nonirritant esters. The model is based on the physicochemical properties of 42 esters that were tested in humans for skin irritation. Nineteen physicochemical parameters that represent transport, electronic, and steric properties were calculated for each chemical. Best subsets regression analysis indicated candidate models for further analysis. Regression analyses identified significant models (p < 0.05) that had variables that were also significant (p < 0.05). These candidate models were evaluated using linear discriminant analysis to determine if the irritant esters could be discriminated from nonirritant esters. The stability of the model was evident from the consistency of parameters among ten submodels generated using multiple random sampling of the database. The sensitivity of the ten models, evaluated by "leave-one-out" cross-validation, ranged from 0.846 to 0.923, with a mean of 0.885 ± 0.025 (95% CI). The specificity ranged from 0.615 to 0.923, with a mean of 0.738 ± 0.06 (CI). Compared with nonirritant esters, irritant esters had lower density, lower water solubility, lower sum of partial positive charges, higher Hansen hydrogen bonding parameter, and higher Hansen dispersion parameter. The results indicate that physicochemical features of esters contribute to their ability to cause skin irritation in humans, and that chemical partitioning into the epidermis and intermolecular reactions are likely important components of the response. This model is applicable for prediction of human irritation of esters yet untested.

Key Words: skin irritation; structure-activity relationship; computational model; organic esters; physicochemical parameters.


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Irritant contact dermatitis affects up to 10% of the U.S. population. It is responsible for more than 18% of all occupational (nonaccident) diseases and more than 90% of occupational skin diseases (Marrakchi and Maibach, 1994Go). Several thousand chemicals, both natural and synthetic, are potentially capable of inducing skin irritation (Marks and DeLeo, 1997Go). Despite this high prevalence, much remains to be learned about the underlying mechanisms involved in the irritant response to chemicals.

Mechanisms of irritation have been identified for homologous series of chemicals. Phorbol esters produce irritation and inflammation through the activation of protein kinase C, which upregulates TNF-{alpha} (Lisby et al., 1995Go). Neurogenic irritants (e.g., capsaicin) cause plasma leakage and vasodilation by inducing the release of neuropeptides (e.g., substance P) from sensory nerves (Baluk, 1997Go). Although there has been substantial progress in identifying pro-inflammatory mediators, such as arachidonic acid, eicosanoids, and IL-1{alpha} produced by skin irritants (Müller-Decker et al., 1998Go), there is limited understanding of the underlying mechanisms of irritation caused by most chemicals.

In the United States and Europe, a rabbit model is used for regulatory purposes to classify chemicals as dermal irritants (Draize et al., 1944Go). The model appears to overestimate the human irritant potential of chemicals, as many predicted by the rabbit model to be strong irritants are weak irritants on human skin (Davies et al., 1972Go; Nixon et al, 1975Go; Phillips et al., 1972Go;). The inadequate performance of the rabbit model in predicting human response and the intention of the European Union to phase out the method has prompted investigation of structure-activity relationship (SAR) models as alternatives to in vivo animal tests.

Several SAR models have been described for skin irritation. The severity of the irritation from organic acids and bases in humans has been found to correlate with the strength of the acid or base, as measured by their pKa (Berner et al., 1988Go; Berner et al., 1990Go; Nangia et al., 1996Go). An SAR model based on rabbit data indicated that irritants have higher log P, lower molecular volume, and higher dipole moment than nonirritants. The author suggested that irritants display enhanced permeability and higher reactivity (Barratt, 1996Go).

The objective of the current study was to use human data to develop an SAR model to predict the irritant response to esters based on their physicochemical properties. Limited multiple random sampling of the database assessed the robustness of the model. The results indicate that physicochemical properties can be used to predict the irritancy of esters.


    MATERIALS AND METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Modeling strategy.
The modeling strategy consisted of several steps. The first and most important was development of the database. The second step was calculation of physicochemical (PC) parameters, followed by regression analysis to identify the relevant PC parameters. Linear discriminant analysis was then used to form the final models to discriminate irritant from nonirritant esters.

Two concerns were addressed: the predictive ability of the model and the consistency of the model form. Subsets of the database were used to generate test models. These were compared to determine if the model forms were consistent. Each of the models so generated is an estimate of the "true model," with the goal being to understand how stable (or consistent) these estimates are when they are derived from random subsets of the database. This procedure helps to indicate the extent to which the true model is dependent on the particular data set used to create it. Multiple random sampling allows one to treat the parent database as an empirical population distribution. As a result, one is able to 1) overcome to some degree the representation bias in the database due to its nonrandom selection and composition; 2) determine the variability and stability of the resulting SAR models derived from random samplings of the empirical distribution; 3) identify the most consistent model parameters from which to build a consensus model; and 4) determine the overall reliability of the final SAR model, that is, the relative dependence of its final form on the data used for its construction.

Database.
The database of chemical irritants was developed from a critical review of the primary literature. Sources of information were the National Library of Medicine's MEDLINE online database; the database of fragrance materials maintained by the Research Institute for Fragrance Materials; the Registry of Toxic Effects of Chemical Substances online database from the National Institute of Occupational Safety and Health; and textbooks on clinical toxicology (Gosselin et al., 1984Go) and contact dermatitis (Rietschel and Fowler, 1995Go). All articles identified through these sources were examined and chemicals were accepted into the database if they met the criteria listed in Table 1Go.


View this table:
[in this window]
[in a new window]
 
TABLE 1 Criteria for Inclusion of Chemicals into the Database and Assignment of Activity
 
Activity of chemicals was assigned on the basis of both the degree of irritant response reported in the study, and the concentration of agent used to produce the response. Responses were considered positive if more than 10% percent of the subjects were reported to have an irritant response and/or the primary irritant index (PII, the numerical grading of irritant skin reactions based on visual observations of erythema/eschar and edema formation) (Patrick and Maibach, 1992Go) was >= 0.5. The PII is the sum of the grade of erythema/eschar formation and the grade of edema formation, each of which is typically graded on a scale of 0 to 4 or 5, with the higher values indicating increased severity. A chemical was considered to be an irritant if the response was observed at a concentration < 20%, and a nonirritant if the response was negative at a concentration >= 20%. Chemicals that were positive only at a concentration > 20%, or that had been tested only at a concentration > 20%, were excluded from the database. Chemicals exhibiting a negative response only at a concentration < 20% were also excluded from the database.

Esters represented the largest class of chemicals within the database (n = 42) and were used for development of the model. The structures of the 13 active esters are shown in Figure 1Go; those of the 29 inactive esters are shown in Figure 2Go.



View larger version (16K):
[in this window]
[in a new window]
 
FIG. 1. Structures of irritant esters that comprised the database.

 


View larger version (25K):
[in this window]
[in a new window]
 
FIG. 2. Structures of nonirritant esters that comprised the database.

 
Physicochemical model.
The molecular modeling and calculation of the physicochemical parameters were performed using Molecular Modeling ProTM (version 1.44; Window Chem Software, Inc.). Individual 3-dimensional structures were drawn and rigid rotor conformational analysis was performed to identify low energy molecular conformations. The global energy minimum structure was subjected to full energy minimization using the MOLY force field (Dyott et al., 1980Go). The energy-minimized structure was submitted to algorithms within Molecular Modeling Pro for calculation of the 19 physicochemical parameters (Table 2Go) that represent electronic, transport, and steric properties. The complete neglect of differential overlap (CNDO) (Pople and Segal, 1965Go, 1966Go) was used to calculate electronic properties such as HOMO, LUMO, partial charges, and dipole moments.


View this table:
[in this window]
[in a new window]
 
TABLE 2 Physicochemical Parameters Examined for Each Chemical
 
Statistical analyses were performed using MINITAB® for Windows Release 11.14. Best subsets regression analysis identified candidate models for subsequent discriminant analysis. The best subsets method identifies regression equations containing from 1 to p parameters where p equals the number of parameters being considered (in this case, 19). These regression equations are then ranked according to R-squared (R2). From the generation of all possible models, all possible subsets of regressors are examined, and those models with the smallest number of regressors, which meet certain statistical criteria, are selected as best. The models here are chosen based on two statistics derived from the model data: Radj2 and the Cp statistic. Radj2 is the multiple correlation coefficient in a form that adjusts for the number of regressors in the model. A large Radj2 implies that much of the variation in the dependent variable (the predicted activity) is accounted for by the model, adjusting for the number of applied regressors incorporated into the model. This implies good prediction of the data used to build the model.

The Cp statistic is useful for selecting a proper subset of regressors. In the situation where there is a large number of possible regressors, the potential exists for overfitting, i.e., too many regressors. This would reduce both the precision of the model coefficients and the future accuracy of the model. Underfitting, i.e., too few regressors, is also a concern that would create bias in the model predictions, the model coefficients, and the estimate of error variance, and also affect future accuracy. Thus a model that is too simple may suffer from biased coefficients and a biased prediction, whereas an overly complicated model can result in large variances both in the coefficients and in prediction.

The Cp statistic for a p-parameter regression model is defined as

where {sigma}2 is estimated by the mean squared error (mse), the residual variance of the full model, i.e., a model that uses all of the regressors, and s2 is the mse of the model with the p regressors in question, i.e., the reduced model. It is apparent from the definition of Cp that if the mse in the reduced model is equal to that of the full model (if s2 = {sigma}2), Cp is a minimum at the value p, the number of regressors in the model. There is nothing to be gained by reducing degrees of freedom, the fitted mse is inflated, by adding more regressors. Therefore, one favors candidate models with the smallest Cp value. The models chosen here were the most parsimonious given the largest Radj2 and smallest Cp (Myers, 1990Go).

Linear discriminant analyses identifies the difference between groups based on a linear combination of variables that are assumed to have a multivariate normal distribution (no dichotomous variables are allowed). The method develops a discriminant function of the form

that has the maximum standard distance between the group means of any possible linear combination of the given variables and represents the model. An untested chemical is classified as active or inactive based on the closeness of its discriminant function value to the group mean based on the function.

The statistical analyses described above were performed on ten subsets of the database. Each subset consisted of the entire set of 13 irritants and 13 randomly selected (multiple random sampling with replacement) of the 29 nonirritant esters. Best subsets regression, linear regression, and linear discriminant analysis was performed on each subset. Sensitivities and specificities were determined by leave-one-out cross-validation.


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
The group of 13 irritant esters was paired with ten random sets (with replacement) of 13 nonirritant esters and statistical analyses were conducted with each set. As indicated in Table 3Go, the most consistent physicochemical parameters in these ten sets were water solubility (occurrence within 5 sets), density (5 sets), Hansen dispersion (5 sets), Hansen hydrogen bonding (5 sets), and sum of partial positive charges (8 sets). Each of the ten sets contained at least one of these five descriptors and the direction of the association of each of these descriptors with activity was consistent (Table 4Go). Irritant esters had lower water solubility and density, higher Hansen dispersion and Hansen hydrogen bonding, and lower sum of partial positive charges compared with nonirritant esters.


View this table:
[in this window]
[in a new window]
 
TABLE 3 SAR Models for Human Irritant Esters
 

View this table:
[in this window]
[in a new window]
 
TABLE 4 Association of Major Parameters with Biological Activity
 
The sensitivity (correct prediction of irritant esters) and specificity (correct prediction of nonirritant esters) of each model was determined using leave-one-out cross-validation. Values are provided in Table 3Go. The sensitivity ranged from 0.846 to 0.923, with a mean of 0.885 ± 0.025 (95% CI); specificity ranged from 0.615 to 0.923, with a mean of 0.738 ± 0.06 (CI). The squared distance represents the degree of separation between the active and inactive group means. For the ten models, the squared distance ranged from 1.6 to 10.2, with a mean value of 4.9 (Table 3Go). Figure 3Go depicts the discriminant functions of models #5 and #7 indicating the squared distance and the degree of overlap of the group functions.



View larger version (13K):
[in this window]
[in a new window]
 
FIG. 3. Frequency histograms of the Z scores from the discriminant function of model #5 (upper) and model #7 (lower). Solid line, nonirritants; broken line, irritants.

 

    DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Although irritant contact dermatitis is a prevalent disease and numerous chemicals are presumed to cause irritation, the literature describing human irritant responses to single chemicals is relatively sparse. A database of human irritants was developed following a critical review of the literature with a rigorous set of acceptance criteria. Consequently, confidence is high in the quality of the data used in this investigation and in the relevance of the findings to human responses.

An irritant should possess the following properties: 1) the ability to penetrate the lipid-rich stratum corneum of the skin, 2) the ability to exit this layer into the epidermis, and 3) the ability to interact with keratinocytes and/or Langerhans cells to release mediators of inflammation such as cytokines or eicosanoids. Given the many factors controlling penetration into the epidermis and reactivity/interaction with cells, there are numerous ways by which a chemical may initiate irritation.

A SAR model of dermal irritation would simplify the regulation of new chemicals proposed for commercial use, and would aid in identifying potential causative agents in situations of multiple chemical exposures. Several models have been reported (Table 5Go). Models for organic acid and base irritants that used human data indicate a correlation of pKa with irritation (Berner et al., 1988Go; Berner et al., 1990Go; Nangia et al., 1996Go). Other models in Table 5Go are based on rabbit data. Barratt (1996) found a diverse set of irritants to have a higher log P, lower molecular weight and molecular volume, and higher dipole moment than nonirritants, and interpreted these findings as indicating greater penetration capacity and reactivity of irritants compared with nonirritants. The sensitivity and specificity of the discriminant analysis was 0.667. The low predictive power was attributed to the inherent variability in the patch test data from which the database was constructed. The stepwise regression indicated a correlation, with R2 = 0.422; cross-validation R2 = 0.201.


View this table:
[in this window]
[in a new window]
 
TABLE 5 SAR Models of Skin Irritation
 
The ester SAR model of dermal irritation presented here differs from most by being derived exclusively from human data, and addressing concerns of model consistency and stability. A goal of SAR modeling is to develop a model that predicts the activity of a large group of chemicals based on the known activity of a subset of such chemicals. Although SAR models typically incorporate data from the largest set of chemicals available, the set is usually nonrepresentative of the universe of chemicals that have the particular toxicity. When chemicals are selected for testing for a discrete toxic endpoint, there is usually a suspicion that they may cause the effect in question. Random selection of chemicals for testing, or testing chemicals suspected to be inactive, is not typical. We sought to address the issue of generalizability by comparing models derived from subsets of the database.

Ten models were generated from the database. Linear discriminant analysis indicated that physicochemical parameters could distinguish irritant from nonirritant esters. For most of the ten models, water solubility, Hansen dispersion, Hansen hydrogen bonding, sum of partial positive charges, and density showed association with activity. In contrast to the findings of Barratt (1996), log P was not a frequent discriminator. However, our finding that irritant esters tended to have lower water solubility is consistent with Barratt's results and suggests that increased dermal penetration favors activity. In addition, the lower density exhibited by the active esters agrees with Barratt's finding that irritants possess lower molecular weight and volume compared with nonirritants.

Our model indicated the importance of parameters such as Hansen dispersion, hydrogen bonding, and the sum of partial positive charges, that are related to intermolecular interactions. Barratt's finding that dipole moment is a significant descriptor may also be interpreted as an intermolecular interaction between ligand and receptor. It must be emphasized that a direct comparison of the performance of our model with that of Barratt is inappropriate due to the different composition of the data bases; ours comprises esters exclusively.

Transport parameters and electronic parameters related to intermolecular interactions were present in most of the ten models indicating that chemical partitioning into tissue layers is likely a prime component of the irritant response. Electronic parameters related to reactivity, such as HOMO and LUMO, were not significant descriptors. LUMO is a global electronic property that relates to the electrophilicity of a compound. Our finding that LUMO does not contribute to discrimination of activity versus inactivity leads us to conclude that electrophilicity is not the rate-determining step for this set of esters.

The stability of the ester model was examined. Stability was apparent from the consistency of descriptors in the ten models, and the direction of association of the descriptors with activity. Only four irritant esters were incorrectly predicted by any one of the models: citronellyl acetate was incorrectly predicted to be negative by nine of the ten models, allyl phenyl acetate was incorrectly predicted by four of the ten models, allyl butyrate and butyl benzyl phthalate each by one model. The sum of partial positive charge parameter appears to be responsible for most of the incorrect predictions. Only one nonirritant ester, benzyl benzoate, was incorrectly predicted to be positive by all of the models. Isobutyl caproate was incorrectly predicted by the majority of the models (4 of 5) of which it was a member. As with the irritant esters, the sum of the positive charges parameter appeared to drive the incorrect predictions.

Use of the Models
When using SAR models for predictive purposes, the goal is to use the most representative characteristics that differentiate irritant from nonirritant esters. Accordingly, all ten models should be used as opposed to using any one submodel. The models may be used in one of two ways. A consensus model can be adopted in which an unknown ester would be judged to be active if it were predicted active by the majority (a consensus) of the ten working models. Alternatively, a combination model can be applied whereby an unknown ester would be judged active if it were predicted active by at least one submodel.

In summary, the results reported here indicate that physicochemical features of certain esters contribute to their ability to cause dermal irritation in humans. The model is directly relevant to human dermal irritation as the data were derived from human studies, rather than from rabbit irritation data, as has been used frequently by others. The features suggest that chemical partitioning into the epidermis and intermolecular reactions are important steps in the irritation process. The features are consistent in models derived from subsets of the data suggesting confidence in an association between these parameters and human skin irritation. Lastly, the study is based on a limited set of chemicals of one class. As such, it is inappropriate to generalize the model to nonesters until validation trials are performed.


    ACKNOWLEDGMENTS
 
This study was supported by NIOSH-CDC #0009653058.


    NOTES
 
1 To whom correspondence should be addressed at Department of Environmental and Occupational Health, University of Pittsburgh, 260 Kappa Drive, Pittsburgh, PA 15238. Fax: (412) 967-6611. E-mail: mhk+{at}pitt.edu. Back


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Baluk, P. (1997). Neurogenic inflammation in skin and airways. J. Investig. Dermatol. Symp. Proc. 2, 76–81.[Medline]

Barratt, M. D. (1996). Quantitative structure-activity relationships for skin irritation and corrosivity of neutral and electrophilic organic chemicals. Toxicol. in Vitro. 10, 247–256.[ISI]

Berner, B., Wilson, D. R., Guy, R. H., Mazzenga, G. C., Clarke, F. H., and Maibach, H. I. (1988). The relationship of pKa and acute skin irritation in man. Pharm. Res. 5, 660–663.[ISI][Medline]

Berner, B., Wilson, D. R., Steffens, R. J., Mazzenga, G. C., Hinz, R., Guy, R. H. and Maibach, H. I. (1990). The relationship between pKa and skin irritation for a series of basic penetrants in man. Fundam. Appl. Toxicol. 15, 760–766.[ISI][Medline]

Bondi, A. (1964). Van der Waals volumes and radii. J. Phys. Chem. 68, 441–451.[ISI]

Davies, R. E., Harper, K. H., and Kynoch, S. R. (1972). Inter-species variation in dermal reactivity. J. Soc. Cosmet. Chem. 23, 371–381.[ISI]

Draize, J. H., Woodard, G., and Calvery, H. O. (1944). Methods for the study of irritation and toxicity of substances applied topically to the skin and mucous membranes. J. Pharm. Exp. Ther. 82, 377–390.

Dyott, T. M., Stuper, A. J., and Zander, G. S. (1980). MOLY-an interactive system for molecular analysis. J. Chem. Inf. Comput. Sci. 20, 28–35.[ISI]

Gosselin, R. E, Smith, R. P., and Hodge, H. C. (1984). Clinical Toxicology of Commercial Products, 5th ed. Williams & Wilkins, Baltimore.

Hansch, C., and Leo, A. (1979). Substituent Constants for Correlation Analysis in Chemistry and Biology. Wiley and Sons, New York.

Klopman, G., Wang, S. and Balthasar, D.M. (1992). Estimation of aqueous solubility of organic molecules by the group contribution approach. Application to the study of biodegradation. J. Chem. Inf. Comput. Sci. 32, 474–482.[ISI][Medline]

Lisby, S., Muller, K. M., Jongeneel, C.V., Saurat, J. H., and Hauser, C. (1995). Nickel and skin irritants up-regulate tumor necrosis factor-alpha mRNA in keratinocytes by different but potentially synergistic mechanisms. Int. Immunol. 7, 343–352.[Abstract]

Marks, J. G., and DeLeo, V. A. (1997). Allergic and irritant contact dermatitis. In Contact and Occupational Dermatology. Mosby, St. Louis.

Marrakchi, S., and Maibach, H. I. (1994). What is occupational contact dermatitis? An operational definition. Dermatol. Clin. 12, 477–484.[ISI][Medline]

McGowan, J. C. (1990). A new approach for the calculation of hydrophile-lipophile balance values of sufactants. Tenside Surfactants 27, 229–230.

Müller-Decker, K., Heinzelmann, T., Fürstenberger, G., Kecskes, A., Lehmann, W.D., and Marks, F. (1998). Arachidonic acid metabolism in primary irritant dermatitis produced by patch testing of human skin with surfactants. Toxicol. Appl. Pharmacol. 153, 59–67.[ISI][Medline]

Myers, R. H. (1990). Classical and Modern Regression with Applications, 2nd ed. PWS-KENT Publishing Company, Boston.

Nangia, A., Andersen, P. H., Berner, B., and Maibach, H. I. (1996). High dissociation constants (pKa) of basic permeants are associated with in vivo skin irritation in man. Contact Dermatitis 34, 237–242.[ISI][Medline]

Nixon, G. A., Tyson, C. A., and Wertz, W. C. (1975). Interspecies comparisons of skin irritancy. Toxicol. Appl. Pharmacol. 31, 481–490.[ISI][Medline]

Patrick, E., and Maibach, H.I. (1992). Predictive assays: animal and man, and in vitro and in vivo. In Contact Dermatitis (R. J. G. Rycroft, T. Menné, P. J. Frosch, and C. Benezra, Eds.), pp. 706–750. Springer-Verlag, Heidelberg.

Phillips, L., Steinberg, M., Maibach, H. I., and Akers, W. A. (1972 ). A comparison of rabbit and human skin responses to certain irritants. Toxicol. Appl. Pharmacol. 21, 369–382.[ISI][Medline]

Pople, J., and Beveridge, D. (1970). Approximate Molecular Orbital Theory. McGraw-Hill.

Pople, J., and Segal, G. A. (1965). Approximate self-consistent molecular orbital theory: II. calculations with complete neglect of differential overlap. J. Chem. Phys. 43, S136–S149.[ISI]

Pople, J. and Segal, G. A. (1966). Approximate self-consistent molecular orbital theory: III. CNDO results for AB(sub)2 and AB(sub)3 systems. J. Chem. Phys. 44, 3289–3296.[ISI]

Rietschel, R. L., and Fowler, J. F. (1995). Fisher's Contact Dermatitis, 4th ed. Williams & Wilkins, Baltimore.

Van Krevelin, D. W. (1990). Properties of Polymers, 3rd edition. Elsevier, Amsterdam.

Viswanadhan, V. N., Ghose, A. K., Revankar, G. R., and Robins, R. K. (1989). Atomic physicochemical parameters for three dimensional structure directed quantitative structure-activity relationships. J. Chem. Inf. Comput. Sci. 29, 163–172.[ISI]





This Article
Abstract
FREE Full Text (PDF)
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Search for citing articles in:
ISI Web of Science (10)
Disclaimer
Request Permissions
Google Scholar
Articles by Smith, J. S.
Articles by Karol, M. H.
PubMed
PubMed Citation
Articles by Smith, J. S.
Articles by Karol, M. H.