1 University of Washington, Seattle, WA.
2 Agency for Healthcare Research and Quality, Bethesda, MD.
3 Centers for Disease Control and Prevention, Atlanta, GA.
4 National Human Genome Research Institute, Bethesda, MD.
5 Foundation for Blood Research, Scarborough, ME.
6 New England Medical Center, Boston, MA.
7 Oregon Health Sciences University, Portland, OR.
8 Baylor College of Medicine, Houston, TX.
9 National Cancer Institute, Bethesda, MD.
10 Case Western Reserve University, Cleveland, OH.
Received for publication September 5, 2001; accepted for publication June 13, 2002.
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
factor V; genetic markers; genetic predisposition to disease; genetic screening; genetics; phenylketonurias
Abbreviations: Abbreviations: HNPCC, hereditary nonpolyposis colorectal cancer; HuGE, Human Genome Epidemiology.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The issues discussed by Little et al. (1) concerning the interpretation of gene-disease associations remain relevant at this transition point, because they influence the interpretation of test results. Similarly, an understanding of any modifying environmental factors that influence the clinical effect of gene variants is important. However, a new issue arises when a genetic test is considered for clinical or public health use: whether the test provides a health benefit.
Because benefit can be evaluated only in the context of specific health outcomes, the starting point for considering the use of a genetic test is a well-defined clinical problem, for which the test is expected to improve care. The test may help direct workup or management of a clinical problem, identify candidates for specific interventions, or provide diagnostic or prognostic information. In particular, genetic testing is expected to provide unparalleled new opportunities for the prevention of disease and adverse drug reactions, through the identification of people with genetic susceptibilities (2, 3).
Over the next decade a growing number of genetic tests will be marketed to clinicians as a result of expanding genetic knowledge. Many of these new tests will use DNA-based technology, but any laboratory test used primarily to identify an inherited condition is considered a genetic test (4). More than 800 genetic tests are currently available or in research development (GeneTests-GeneClinics; http://www.geneclinics.org). Although most currently available tests are for rare diseases, tests to identify inherited risk for common diseases such as breast and colorectal cancer, thromboembolism, Alzheimers disease, and coronary heart disease have been developed.
As new genetic tests become available, policy makers, clinicians, and the public will need to make decisions about test use. New tests are likely to vary in their predictive value, their potential to direct prevention or treatment efforts, and their personal and social consequences. Therefore, the task of determining the appropriate use for a given test will require careful consideration of a variety of factors. In developing practice guidelines, different professional organizations, agencies, and health care systems will utilize a range of methodologies to evaluate evidence and reach final decisions about the indications for test use (57). Guidelines procedures also vary in their consideration of costs and other social issues, in part because of differences in health care priorities and resources in different health care settings. However, all practice guidelines procedures require clear and accurate information about the test under consideration. In addition, the appropriate implementation of most practice guidelines involves clinical judgment and consideration of patient preferences; thus, clinicians and patients also need accessible and accurate information about genetic test properties. To aid policy makers, clinicians, and the public in the prudent use of new genetic tests, we propose a standardized approach to reporting the data defining test properties. Our discussion of the issues to be addressed in the reporting process is illustrated by examples of genetic tests with high and low predictive value.
![]() |
PHENYLKETONURIA: HIGH EXPECTATIONS FOR IMPROVED PREVENTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The success of newborn screening for phenylketonuria can be attributed to four factors: a reliable and accurate screening process; a readily identifiable population for screening; an effective treatment for those who test positive; and timely initiation of treatment (8). The current phenylketonuria screening protocol, which includes follow-up testing for confirmation after an initial elevated phenylalanine level, leads to highly sensitive and specific identification of children with phenylketonuria. However, initial screening efforts did not have this level of accuracy. In the initial phases of phenylketonuria screening, some children with moderate phenylalanine elevations were classified as affected and suffered adverse consequences from a phenylalanine-restricted diet (9).
Further, although the phenylalanine-restricted diet is effective in preventing mental retardation, it is not without burdens. Children must be taught to avoid many common foods and to adhere to a diet that is not palatable to most people. Social burdens are involved; a child with phenylketonuria must be taught to avoid the hamburgers and ice cream that may be offered at a friends house and to handle the inevitable explanations. Education and follow-up are needed to enhance compliance, with special attention to the transition to self-management that occurs in adolescence. In addition, subsidized programs for disadvantaged families are necessary to ensure that the diet is available to all newborns that test positive.
Finally, it was not until newborn screening for phenylketonuria had been instituted that an additional complication of phenylketonuria was identified: microcephaly in children of mothers affected with phenylketonuria whose phenylalanine levels were elevated during pregnancy. This outcome led to recognition of the need for tight dietary control before and during pregnancy in women with phenylketonuria. Similarly, ongoing follow-up has demonstrated that all patients with phenylketonuria benefit from continued dietary control in adulthood, illustrating the long-term social consequences of this disorder.
These experiences emphasize that data on a broad range of issues are relevant to the evaluation of a genetic test. These include not only the expected clinical measuresthe reliability of the laboratory measurement, the predictive value of the test, and the effectiveness of medical interventionsbut also social issues such as access to care and the personal and familial implications of testing and interventions. The phenylketonuria experience also underscores the need for ongoing data collection after a test is introduced to identify unanticipated consequences of the testing process.
![]() |
GENETIC TESTS WITH LIMITED PREDICTIVE VALUE: FACTOR V LEIDEN AND VENOUS THROMBOEMBOLISM |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Factor V Leiden, the DNA-based test most commonly ordered in the United States, offers an example to illustrate these issues. This genetic trait is a variant of the factor V gene, which codes for a protein involved in clot formation. It is present in approximately 5 percent of European Americans, 2 percent of Hispanic Americans, and 1 percent of African and Asian Americans (12). Many studies, done predominantly in populations of European descent, have now documented its association with an increased risk of venous thromboembolism, often in the setting of other circumstances that affect risk, including pregnancy, use of oral contraceptives, and surgery (1216). Estimates of the increased risk for venous thromboembolism among carriers of factor V Leiden vary from two- to eightfold (16, 17).
Factor V Leiden is one of several genetic factors that contribute to risk for venous thromboembolism (16), and interactions between genetic and nongenetic risk factors for venous thromboembolism are well established (16, 18). For example, individuals carrying both factor V Leiden and the 202210A variant in the prothrombin gene have a substantially higher risk of venous thromboembolism than do carriers of either trait alone (17, 19, 20). In more than half of a series of families with antithrombin III deficiency, another genetic risk factor, the high risk of venous thromboembolism was due to the additional presence of factor V Leiden or the prothrombin variant (21). Thus, when venous thromboembolism occurs in the presence of factor V Leiden, it oftenperhaps alwaysrepresents the effect of interactions with one or more other genetic and nongenetic risk factors (16). Estimates of the annual incidence of venous thromboembolism events in people with factor V Leiden range from 0.19 percent to 0.58 percent (13, 18, 20), suggesting that cumulative risk is in the range of 1030 percent.
Although factor V Leiden is well established as a risk factor, its implications for clinical management are far from clear. Several rationales for factor V Leiden screening have been proposed. Carriers of the variant might benefit from avoiding oral contraception (14, 22) or postmenopausal estrogen (23). Anticoagulant prophylaxis might reduce the risk of venous thromboembolism when transient risk factors, such as surgery, pregnancy, or prolonged bed rest, occur (16, 20, 22). However, these interventions all pose potential risks as well as benefits, and none has been systematically studied in people with factor V Leiden.
These various possibilities demonstrate the importance of focusing on a defined clinical problem and setting when determining the evidence needed to assess test use. Studies assessing different test purposes require different testing protocols, study populations, and outcome measures. For example, an assessment of the use of factor V Leiden testing prior to oral contraceptive use would require a population of women receiving contraceptives and would need to evaluate both contraceptive-related thrombotic risk and the outcome of alternative methods of contraception. Conversely, the assessment of testing to determine the appropriate thrombosis prophylaxis prior to surgery would rely on the measurement of postoperative outcomes in surgical patients using specified prophylaxis protocols. Each study would require appropriate control populations and measurement of other factors associated with the risk of venous thromboembolism. Without such studies, the clinical utility of testing for factor V Leiden is called into question.
![]() |
DEVELOPING STANDARDIZED APPROACHES TO PRESENTING EVIDENCE |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Analytic validity
The term analytic validity refers to the accuracy with which a particular genetic characteristic (e.g., a DNA sequence variant) can be identified in a given laboratory test. One way to report analytic validity is in terms of the tests sensitivity and specificity for the genetic variant in question. Most genetic variants can be tested by a variety of protocols, and a number of technical issues arise in evaluating analytic validity. These include the assay chosen, the reliability of the assay, the degree to which reliability varies from laboratory to laboratory, and the complexity of test interpretation. Thus, an oligonucleotide probe for a single nucleotide sequence variant is a simpler test than a linkage analysis. In the latter, accuracy of the test is based on the accuracy with which samples and medical history are collected from family members, as well as on technical aspects of the testing process in the laboratory.
Often, few published data are available concerning the analytic validity of a test, and it may be difficult to judge whether different studies of the same genetic trait used comparable laboratory methods. For example, some studies of factor V Leiden utilize a DNA-based assay while others utilize a functional assay, with differences in sensitivity and specificity (22). An adequate description of the analytic validity of a test requires systematic collection of data for this purpose, using defined populations with known genotypes (table 1). Proficiency testing, such as that performed under the auspices of the American College of Medical Genetics and the College of American Pathologists, provides one source of objective data on analytic validity, but proficiency testing usually involves small numbers and few studies of normal genotypes. The scarcity of published studies on analytic validity is an important limitation in the available evidence about genetic tests.
|
Clinical validity is often uncertain for new genetic tests. Estimates may be based on studies in limited and potentially biased populations. Uncertainty about clinical validity is an important consideration for policy makers, clinicians, and patients, particularly when they are deciding whether the risk information provided by the test provides a sufficient rationale for the use of interventions with putative but unproven benefits.
Mutations associated with disease susceptibility are often first defined in "high risk" families, characterized by multiple affected family members, and these studies may overestimate risk. The BRCA1 and BRCA2 genes, for example, were first identified in families selected for the presence of breast cancer in women aged less than 60 years or ovarian cancer in four or more family members (24). The lifetime risk of breast cancer is estimated to be about 85 percent in mutation carriers from these high-risk families. However, studies using less selected populations have estimated the lifetime risk associated with BRCA1 and BRAC2 mutations to be 3656 percent (2528). These observations suggest that high-risk families represent the severe end of a spectrum of risk associated with BRCA1 and BRAC2 mutations and that other factors, both genetic and nongenetic, may modify risk (29). Thus, variation in clinical validity may be due to complex gene-gene and gene-environment interactions, with the result that the predictive value of many genetic tests may remain imprecise even when all appropriate studies have been completed.
Important variables in evaluating evidence about clinical validity include the size and selection criteria for the study population, the type of test used (and its analytic validity), the study design, and the clinical and other endpoints measured (table 1). Careful definition of the measures used to define the clinical outcome is needed. Studies will provide more convincing results when they also include measurement of nongenetic factors that contribute to the clinical outcome. Key methodological considerations include the comparability of case and control populations (where a case-control design is used), whether the interpretation of endpoints was blinded, and whether negative results were verified.
Many genetic variants affect more than one clinical outcome; for example, factor V Leiden is associated with both venous thromboembolism and pregnancy loss (30), and some genetic variants increase risk for two or more different cancers (24). Multiple disease endpoints need to be considered when studying such variants; clinical validity may be high for one endpoint and low for another.
Clinical utility
Clinical utility refers to the likelihood that the test will lead to an improved health outcome. Measurement of this characteristic requires evaluation of outcomes associated with testing and clinical interventions. Central questions are the effectiveness of the interventions available for people who test positive and the social consequences of testing for people with positive and negative test results. Standardized approaches are available to assess the quality of evidence concerning interventions (31).
As with information about clinical validity, knowledge about optimal prevention strategies is accumulated incrementally. As is usual with emerging technology, people most genetically susceptible are initially offered interventions based on clinical reasoning or extrapolation from data in other populations, for example, early initiation of mammography screening in women with BRCA1 and BRAC2 mutations. The risks and benefits of these interventions can be understood only after systematic observation, in the form of well-designed controlled trials, cohort studies, or case-control studies. Because most genetic risk factors, even for common diseases, occur in a small percentage of the population (e.g., the 15 percent prevalence of factor V Leiden), sample sizes of genetically susceptible subjects are often small. Definitive understanding of clinical utility may be dependent on reliable methods for the pooled analysis of different studies.
Important variables in evaluating evidence about clinical utility are the size and selection criteria for the study population, the type of laboratory assay and intervention used, and the study design (table 1). The health benefit proposed for a test will determine some of the outcome measures; others are determined by hypotheses concerning the harms of the testing process. As with clinical validity studies, the comparability of the cases and controls is an important issue when a case-control design is used. If an experimental design is used, elements of the study methodology (e.g., randomization strategy) are important. The methods for measuring clinical and other endpoints should be specified, as well as whether negative results were verified.
Because genetic variants may affect multiple different clinical endpoints, a test that has high clinical utility for one endpoint may also provide extraneous information of little utility for another. For example, a person with a mutation predisposing to hereditary nonpolyposis colorectal cancer (HNPCC) faces a high lifetime risk of colorectal cancer and is likely to benefit from early initiation of regular colonoscopy as a means to reduce colorectal cancer mortality (32). However, HNPCC is also associated with increased risk for endometrial, ureteral tract, and upper gastrointestinal cancers; methods of screening for endometrial cancer are of unproven efficacy, and no preventive care is available for the other cancers associated with HNPCC (33).
![]() |
ETHICAL, LEGAL, AND SOCIAL IMPLICATIONS OF GENETIC TESTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Unfortunately, the outcomes that have generated the greatest concern about genetic testing, such as insurance or employment discrimination, stigmatization, and long-term psychologic harms from testing (3436), are difficult to study. Separate studies designed to provide adequate measurements of these outcomes may be necessary. For example, a United Kingdom survey evaluated reports of insurance discrimination among families with a wide range of genetic disorders (37). Similarly, evaluation of costs and of inequities in access to genetic services may require studies designed to examine this issue.
Understanding the factors that determine interest in predictive testing is also important. The clinical utility and personal value attached to knowledge about genetic risk may vary by age or other personal characteristics. For example, the implications of a positive test for a BRCA1 or BRCA2 mutation differ considerably for a woman who has not yet had children compared with one who has, because oophorectomy is an important prevention option for such women. In addition, some testing decisions may be motivated by the desire to help children; a woman with cancer may be more interested in BRCA1 and BRAC2 testing if she has daughters who might benefit from the information.
The appropriate application of genetic testing requires information about the costs and effects of different pretest and posttest counseling procedures and about posttest behaviors. Knowledge about genetic risk does not necessarily lead to risk-reducing behaviors and could, contrariwise, produce a fatalistic attitude that reduces motivation (38). These concerns point to the importance of studying the meaning assigned to genetic information (e.g., whether test recipients or health care providers view genetic test results differently from the results of other medical tests). For example, the media often represent genetics as deterministic (39), but whether this misunderstanding affects the interpretation of genetic test results is not known. Some experimental study designs can incorporate these empirical questions, but long-term follow-up may be needed to ensure an adequate understanding of the personal and social outcomes of the testing process.
![]() |
GENETIC TEST REPORTS: MAKING SENSE OF THE EVIDENCE |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The first step in this process is to encourage authors to present methods and results in standardized ways, as outlined in table 1 and in Little et al. (1). Brief summaries following a succinct and systematic format similar in concept to a structured abstract would provide information about the current status of genetic tests. Because knowledge about tests will change over time, these status reports would ideally be dated and revised on a periodic basis.
Two other more detailed types of reports can provide the necessary background for the brief summaries and offer additional support to policy makers, clinicians, and researchers. One is the structured review, providing a systematic evaluation of evidence on the analytic validity, clinical validity, and clinical utility of a test, including consideration of the ethical, legal, and social implications of testing. The second is the construction of evidence tables that provide ready access to information about the individual studies on which summaries or structured reviews are based. An example of an evidence table is shown in table 2. A reporting format of this kind clarifies both the strengths and the weaknesses of available data, allows studies to be compared, and makes gaps in knowledge apparent. In the hypothetical screening study shown in table 2, for example, the evidence table makes the weaknesses of the study apparent (e.g., intervention/control status determined by patient preference; lack of assessment of analytic validity or quality of life; lack of confirmation of negative screening results) and suggests that screening in this example increases hospitalizations but does not reduce mortality over a 5-year follow-up period. In addition to providing a realistic picture of what is known about a genetic test, an evidence table identifies additional research needs and facilitates the selection of studies for pooled analyses. This format can also be readily updated by the addition of new studies to the table.
|
These strategies for reporting information about genetic tests should not be viewed as replacements for clinical practice guidelines. The latter address the use of a genetic test in a particular health care setting and must take into account the value of testing for the population being served, its cost and acceptability in this setting, and its priority relative to other health care services. Tests suitable in some settings may not be suitable in others. Although procedures beyond the test reporting outlined here are needed to address these issues, objective information concerning the current state of knowledge about a test is an essential component of this process.
![]() |
SUMMARY |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
This ambitious research agenda is unlikely to be achieved rapidly or completely for most genetic tests. Even when all appropriate research is completed, the predictive value of many genetic tests may remain imprecise, and the clinical utility may be variable. As a result, methods to disseminate information about genetic tests represent an important strategy for ensuring appropriate test use. Following criteria for the objective assessment of test properties, reports should be structured to enable policy makers, clinicians, and the public to identify quickly what is known and not known about analytic validity, clinical validity, and clinical utility, so that uncertainties can be taken into account when genetic tests are considered for use.
![]() |
ACKNOWLEDGMENTS |
---|
![]() |
NOTES |
---|
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Related articles in Am. J. Epidemiol.: