1 Monash Institute of Health Services Research, Monash University, Clayton 3168, Australia
2 To whom correspondence should be addressed at: Monash Institute of Health Services Research, Locked Bag 29, Monash Medical Centre, Clayton VIC 3168, Australia. e-mail: dragan.ilic{at}med.monash.edu.au
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Keywords: ADAM/androgen deficiency/Internet/quality of information
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Internet access to medical information, support networks and the purchase of medications online provides consumers with the opportunity to actively participate in decision-making processes, including screening and treatment procedures, that affect their health outcome (Silberg et al., 1997; Sacchetti et al., 1999
; Paulter et al., 2001
). Despite the potential benefit of such interactions, concerns remain regarding the quality of information provided on websites (Jadad and Gagliardi, 1998
). The deregulated nature of the Internet provides a vehicle for anyone wishing to publish health-related information, evidence-based or anecdotal. Although copyright, libel and privacy laws exist, their interpretation and statutes may often differ between countries (Goudreau, 1999
). The Internet represents a liberated medium that operates across geographical borders where national laws become increasingly difficult to enforce and have limited jurisdiction. Therefore without governance the quality of online information remains largely uncontrolled and has the potential to negatively influence consumer health outcomes.
In the pursuit of online health information, consumers face the concurrent challenges of finding the information and of determining its quality. Search-engines provide an index of the Internet, allowing an easy and efficient means of accessing and filtering vast amounts of information (Tay et al., 1998; Wukovitz, 2001
). Although hundreds of general search-engines are available, finding relevant and valid health information remains difficult due to the structure and size of the Internet. Specialized medical search-engines have been developed with the aim to make searching easier by identifying websites, according to established criteria, to provide a specific index with medical or healthcare content (Wukovitz, 2001
). However, even the most efficient search-engine cannot assess the validity and quality of the information, leaving the consumer to evaluate information independently.
Quality of information-rating tools, codes and logos have been developed to provide consumers with the opportunity to identify and appraise the quality of online information (Jadad and Gagliardi, 1998). The majority of these tools however, are not specifically designed to evaluate medical information. Most are concerned with the design and presentation of information with few containing validated criteria for quality assessment of that information (Jadad and Gagliardi, 1998
). Tools such as DISCERN (Charnock et al., 1999
) and the Health Summit Working Group IQ tool (Health Summit Working Group, 2001
, accessed on December 11, 2001) have been developed to specifically appraise the quality of medical website information (Charnock et al., 1999
), allowing the consumer to differentiate between websites containing information they can believe and websites with invalid information.
Although studies assessing Internet use and the quality of information have been performed on various health topics (Impicciatore et al., 1997; Graber et al., 1999
; Latthe et al., 1999
, 2000; Sacchetti et al., 1999
; Okamura et al., 2002
) none has assessed the nature of information regarding androgen deficiency in the ageing male (ADAM). We aimed to compare the search efficiency and the quality of the information available between general search and medical search-engines. We also aimed to determine whether any website characteristics were associated with a higher quality of website information. We chose to assess this health topic (i.e. ADAM) because of the current debate surrounding its physiological validity within the medical community and increasing consumer interest regarding treatment regimes. We also hypothesized that it is a topic that consumers may choose to investigate with the anonymity of the Internet.
![]() |
Materials and methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Websites were considered to be relevant if they provided sufficient information to answer the question, "How is ADAM recognized as a medical condition and what treatment regimes are available?" In particular we sought information describing the background of the condition, its symptoms, diagnosis and treatment options. Relevant resources included medical or general information websites, online journals and news reports. We excluded websites featuring information on hypogonadism as it is considered a condition independent of ADAM (Petak et al., 2002). Websites were excluded from the analysis if they were repeated under a different URL, were a dead link, the server was unavailable or had password requirements, and those in a language other than English.
The quality of website information was assessed using the DISCERN quality assessment tool (Charnock et al., 1999) The DISCERN quality assessment tool rates the quality of website information on a five point Likert scale (1 = "the publication is poor quality and has serious shortcomings and is not a useful or appropriate source of information about treatment choices", 5 = "the publication is good quality and is a useful and appropriate source of information about treatment choices") (Charnock, 1998
). Whilst originally validated to assess written patient information with respect to treatment options, it has since been modified for consumer use in validating health information on the Internet.
Overall search-engine efficiency was measured as the percentage of relevant websites identified by each search-engine from the total number of websites identified for each search query. The efficiency of search-engines in retrieval of quality website information was assessed in addition to the presence and absence of predetermined website characteristics including target audience, authorship and evidence citation to assess if these easily identified criteria were associated with a higher quality of information rating. Analysis of all statistical parameters was performed using a one-tailed MannWhitney U-test.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
The current evidence underpinning the condition of ADAM is poor. There are no recognized clinical guidelines rationalizing a standard diagnosis or treatment procedure for ADAM. Despite this lack of evidence, 19 of the 52 websites recommended treatment with testosterone replacement therapy (TRT), two opposed the use of TRT, 24 provided neutral information relaying the benefits and risks associated with TRT and seven recommended alternative therapies. Evidence supporting website recommendations were cited in seven of the 19 websites recommending TRT, seven of the 24 providing neutral information and two of the seven websites recommending alternate therapies. No evidence was cited in the two websites opposing TRT.
The website characteristics identified, which, when present, resulted in a significantly better quality of website information as assessed by DISCERN, included websites with a "target group identified", "links to the website from other websites" and "citation of the evidence" (Table III). Approval of website information from central organizations such as HON attempt to guide consumers to websites with a better quality of information. This study identified that there was no significant difference between websites with external approval and those without (P = 0.21) (Table III).
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Consumers are turning to the Internet in an attempt to obtain information that will allow them to have a greater role in decision-making processes affecting their health outcomes. Although search-engines are the preferred method of identifying information on the Internet, their use does not guarantee the retrieval of relevant websites. Irrelevant websites often obtain a high ranking due to the manipulation of website characteristics or payment of website placement by authors (Lissman and Boehnlein, 2001). Despite the substantial undertaking by commercial and government entities, the creation of medical search-engines does not appear to provide a more specific index of health websites relating to ADAM. Coupled with findings identifying that consumers prefer to use general rather than medical search-engines when searching for medical information (Eysenbach and Kohler, 2002
), further research is needed to investigate the relative value of medical search-engine development.
Consumers are often presented with conflicting online information, particularly in topics with a poor evidence base. Current tools raise consumer awareness regarding quality issues but do not specifically address the evidence underpinning online information. Invariably consumers are left to their own devices to make an informed decision as these tools lack the power to discriminate between the treatment recommendations of websites and those supported by validated evidence. However, the majority of consumers lack the skills to appraise information critically and identify website characteristics that define the quality of the presented information, hence their reliance on a third party and the continuance of this cycle.
Our results support previous studies that have identified a lack of quality online health information across various topics including reproductive health (Impicciatore et al., 1997; Graber et al., 1999
; Latthe et al., 1999
, 2000; Okamura et al., 2002
). This is in contrast with findings by a previous study that identified complete and up to date online information regarding the treatment of erectile dysfunction (Sacchetti et al., 1999
). Such a contrast in the quality of online information may be representative of a better evidence base highlighting the current disparity of information available within topics concerning male reproductive health. Our findings also lend support to a previous study suggesting that medical search-engines are not as successful in returning relevant medical information in comparison with general search-engines (Graber et al., 1999
).
Due to the constantly evolving nature of the Internet, these results can only represent a snap shot of the Internet at a given point in time. The medical validity of ADAM is contentious and it is not possible to extrapolate these findings to assume that medical search-engines are inefficient in retrieving relevant websites across all health topics and do not provide an equal or better retrieval of quality website information in other areas. Future studies could investigate whether this trend is apparent in other mens health issues but should not only examine the quality of website information but whether evidence supports the website treatment recommendations. Future replication of this study could also identify whether a possible change in the medical stance on ADAM existence is reflected in a change in website information and its quality.
Until websites begin to cite evidence-based material, the quality of online information will remain variable, especially in topics such as ADAM in which there is limited evidence. When searching for the best quality online information, sourcing several websites and relying on those identifying a target group, citing references and linking to other websites may increase validity.
In conclusion, although the quality of online information regarding ADAM is shown to be poor, it is apparent that medical search-engines are no better than general search-engines in sourcing for such information. Approval from external bodies does not necessarily assure a better quality of website information whilst current quality assessment tools can only raise consumer awareness regarding website information. Future tools must evaluate whether treatment recommendations are supported by current evidence in the next step to aid consumers in their search for quality online information.
![]() |
Acknowledgement |
---|
![]() |
FOOTNOTES |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Charnock, D., Shepperd, S., Needham, G. and Gann, R. (1999) DISCERN: An instrument for judging the quality of written consumer health information on treatment choices. J. Epidemiol. Community Health, 53, 105111.[Abstract]
Computer Industry Almanac (2002) Available at: http://www.c-i-a.com/pr032102.htm.
Eysenbach, G. and Kohler, C. (2002) How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. Br. Med. J., 324, 573577.
Goudreau, K. (1999) The copyright quagmire on the Internet. Comput. Nurs., 17, 8285.[Medline]
Graber, M., Bergus, G.R. and York, C. (1999) Using the World Wide Web to answer clinical questions: How efficient are different methods of information retrieval? J. Fam. Pract., 48, 520524.[ISI][Medline]
Health Summit Working Group Information (2001) Available at: http://hitiweb.mitretek.org/iq.
Impicciatore, P., Pandolfini, C., Casella, N. and Bonati, M. (1997) Reliability of health information for the public on the world wide web: Systematic survey of advice on managing fever in children at home. Br. Med. J., 314, 18751878.
Jadad, A. and Gagliardi, A. (1998) Rating health information on the Internet. Navigating to knowledge or to babel? JAMA, 279, 611614.
Latthe, M., Latthe, P.M. and Charlton, R. (1999) Quality of information on emergency contraception on the Internet. Brit. J. Fam. Plann., 26, 3943.[ISI]
Latthe, P., Latthe, M. and Khan, K.S. (2000) Quality of medical information about menorrhagia on the worldwide web. Brit. J. Obstet. Gynecol., 107, 3943.[ISI]
Lawrence, S. and Giles, CL. (1998) Searching the world wide web. Science, 280, 98100.
Lissman, T. and Boehnlein, J.K. (2001) A critical review of internet information about depression. Psychiatr. Serv., 52, 10461050.
Okamura, M., Bernstein, J. and Fidler, A.T. (2002) Assessing the quality of infertility resources on the world wide web: Tools to guide clients through the maze of fact and fiction. J. Midwifery Womens Health, 47, 264268.[CrossRef][ISI][Medline]
Paulter, S., Tan, J.K., Dugas, G.R., Pus, N., Ferri, M., Hardie, W.R. and Chin, J.L. (2001) Use of the Internet for self-education by patients with prostate cancer. Urology, 57, 230233.
Petak, S.M., Baskin, H.J., Bergman, D.A., Dickey, R.A. and Nankin, H.R. (2002) ACCE medical guidelines for clinicl practice for the evaluation and treatment of hypogonadism in adult male patients. Endocrine Pract., 8, 439456.
Sacchetti, P., Zvara, P. and Plante, M.K. (1999) The Internet and patient education resources and their reliability: Focus on a select urologic topic. Urology, 53, 11171120.[CrossRef][ISI][Medline]
SearchEngineWatch (2002) Available at: http://www.searchenginewatch.com/links/major.html.
Silberg, W., Lundberg, G.D. and Musacchio, R.A. (1997) Assessing, controlling, and assuring the quality of medical information on the Internet. JAMA, 277, 12441245.[CrossRef][ISI][Medline]
Tay, J., Ke, S. and Lun, K.C. (1998) MediAgent: A WWW-based scalable and self-learning medical search engine. Proc. AMIA Symp., 597601.
Wukovitz, L. (2001) Using Internet search engines and library catalogs to locate toxicology information. Toxicology, 157, 121139.[CrossRef][ISI][Medline]
Submitted on September 23, 2002; resubmitted on December 4, 2002; accepted on December 12, 2002.