Accessibility, nature and quality of health information on the Internet: a survey on osteoarthritis

S. Maloney, D. Ilic and S. Green

Monash Institute of Health Services Research, Monash University, Clayton, Victoria 3168, Australia.

Correspondence to: D. Ilic, Monash Institute of Health Services Research, Level 1 Block E, Monash Medical Centre, Clayton, VIC 3168, Australia. E-mail: dragan.ilic{at}med.monash.edu.au


    Abstract
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Objectives. This study aims to determine the quality and validity of information available on the Internet about osteoarthritis and to investigate the best way of sourcing this information.

Methods. Keywords relevant to osteoarthritis were searched across 15 search engines representing medical, general and meta-search engines. Search engine efficiency was defined as the percentage of unique and relevant websites from all websites returned by each search engine. The quality of relevant information was appraised using the DISCERN tool and the concordance of the information offered by the website with the available evidence about osteoarthritis determined.

Results. A total of 3443 websites were retrieved, of which 344 were identified as unique and providing information relevant to osteoarthritis. The overall quality of website information was poor. There was no significant difference between types of search engine in sourcing relevant information; however, the information retrieved from medical search engines was of a higher quality. Fewer than a third of the websites identified as offering relevant information cited evidence to support their recommendations.

Conclusions. Although the overall quality of website information about osteoarthritis was poor, medical search engines may provide consumers with the opportunity to source high-quality health information on the Internet. In the era of evidence-based medicine, one of the main obstacles to the Internet reaching its potential as a medical resource is the failure of websites to incorporate and attribute evidence-based information.

KEY WORDS: Osteoarthritis, Quality of information, Internet, e-Health, Patient education


    Introduction
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Osteoarthritis is the most common of the rheumatic diseases. With an estimated 40 000 new cases of osteoarthritis diagnosed each year, it is the third leading cause of life-years lost due to disability and carries high morbidity and health-care utilization [1, 2]. The range of treatments for osteoarthritis is continually increasing as conventional therapies, such as pharmaceutical management, physiotherapy and joint replacement surgery, are coupled with emerging and established complementary therapies. Health-care providers and people with osteoarthritis are faced with many management decisions, and making these decisions requires valid information. Consumer education is essential to inform shared clinical decision-making and is of paramount importance in a chronic disease with many treatment options.

The Internet is continually evolving as a global communications network. Several hundred million people worldwide use the Internet, of whom 37% use it as a source of health information [3, 4]. It is not only health-care consumers who are turning to the Internet as a resource for self-education; clinicians are increasingly relying on the Internet to update their clinical knowledge [5]. The speed with which new technologies and treatments are being developed means that many clinicians are dependent on the Internet to access information to guide their practice. This is particularly the case for those in rural and remote areas. The use of the Internet to inform and influence health care, termed e-health, has significant potential to inform health-care decisions, improve health management and produce better patient outcomes.

General search engines have been designed to help source information on the Internet. Meta-search engines have been developed in the attempt to provide a more specific return of results from the initial search query by performing simultaneous searches across designated general search engines and collating the results. Medical search engines attempt to locate website information relevant to users’ needs by only indexing online information within a medical context. Subsequently, the medical search engine index is not as large as general search engines and may help consumers identify relevant information.

The greatest barrier to the Internet reaching its potential to inform health care is not the difficulty in finding information, rather finding valid, reliable information. Many factors affect the quality of web-based information. Lack of governance is both a strength and weakness of the Internet as a communications system. Proprietors of health-care websites are competing for sales and market share, which can often lead to selective disclosure of evidence and the presentation of inaccurate information. Unlike print media, there is no editorial control to protect consumers. Additionally, website creators can influence search results through sponsorship of search engines or by manipulating their website characteristics to increase their ‘visibility’ to the searching engine.

Several initiatives have been introduced in an attempt to improve the quality of Internet-based health information. Organizational controls such as the Health on the Net (HON) code offer a certain form of regulation by providing a ‘stamp of approval’ for websites adhering to agreed quality principles. Similar models include the rating of website information by independent third parties offering consumers access to ‘filtered’ quality information. The main drawback facing these initiatives is that new information appears faster than it can be appraised due to the decentralized nature and continued growth of the Internet [6]. An alternative form of control is to place greater responsibility on the Internet user by providing access to tools, such as the DISCERN tool, which gives a mechanism to consumers for appraising web-based health information, up-skilling and empowering them to recognize valid and reliable information [7–9].

This study aims to evaluate the Internet as a source of information for patients and consumers about osteoarthritis by (i) determining the comparative efficiency of medical, general and meta-search engines in retrieving relevant information and (ii) assessing the quality of the relevant information retrieved, and its concordance with the evidence base underpinning the management of osteoarthritis.

As this was a study of published information and involved no participants no ethics approval was required.


    Methods
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Identification of websites
A key-word search of the Internet was performed between October and December 2003. Key words were selected to represent terms consumers may use when searching for information about osteoarthritis on the Internet. Key words used in the search were: arthritis, osteoarthritis, degenerative arthritis, degenerative joint disease and osteoarthrosis. These key words were entered in 15 different search engines, five each of medical, general and meta-search engines (Table 1). Searches were not restricted in terms of file format or domain. The first 50 websites returned by each search engine were examined, as individuals rarely examine beyond 50 sites when sourcing online information [10].


View this table:
[in this window]
[in a new window]
 
TABLE 1. Number of relevant and total websites identified across individual general, meta and medical search engines and the mean (±S.D.) DISCERN quality score of websites retrieved by search engine

 
Websites were considered relevant if they provided sufficient information to answer the question ‘How is osteoarthritis diagnosed and/or what treatment regimes are available?’. Websites were excluded if (i) they were already examined as repeated under a different uniform resource locator (URL), (ii) were presented in a language other than English, (iii) denied direct access through password requirements or repeated server unavailability, (iv) were a library, gateway or portal, (v) provided information about osteoarthritis in animals, or (vi) sold osteoarthritis therapies without providing any information on the topic itself. All unique, relevant websites were saved on DVD for later analysis to avoid any changes that may be made to the selected websites during the period of analysis.

Assessment of website quality
The quality of website information was assessed by one investigator using the DISCERN tool. To ensure reliability, a subset of websites was assessed by a second investigator. The DISCERN tool has been designed to help consumers appraise health information on the Internet. The tool examines website information against 16 key criteria, resulting in an overall assessment of quality on a five-point Likert scale ranging from 1 (the publication is poor) to 5 (the publication is ‘good’ quality and is a useful and appropriate source of information about treatment choices) [7, 8]. Websites were also assessed for the presence or absence of 10 key criteria, previously identified as indicators of quality website information (Table 2) [11, 12]. Whether websites cited evidence, and the level of evidence used to underpin any recommendations (based on the National Health and Medical Research Council levels of evidence) was extracted [13]. The target audience of the website (consumer, health professional or both) was extracted.


View this table:
[in this window]
[in a new window]
 
TABLE 2. Website quality associated with the presence/absence of website characteristics

 
Analysis and data interpretation
Overall search engine efficiency was defined as the number of unique and relevant websites returned as a percentage of the overall number of websites returned. A one-way ANOVA was performed to determine whether the quality of website information (DISCERN score) differed among websites returned from medical, general and meta-search engines. Unpaired t-tests were used to determine the significance of the presence/absence of each website characteristic with respect to quality of information. A comparison between the quality of information of website target audience (professional or consumer) was made using a one-way ANOVA, with a Tukey–Kramer multiple comparisons test being used post hoc. Descriptive statistics (percentages) were used to summarize the proportion of websites making evidence-based recommendations.


    Results
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
A total of 3443 websites were retrieved, of which 344 unique websites contained information relevant to osteoarthritis. Of the 344 individual, relevant websites identified, 103 (29.9%) cited evidence to support the information presented, of which 69 (20.1%) drew on evidence from systematic reviews and randomized controlled trials. The majority of websites (37.5%) recommended a multidisciplinary approach to the treatment of osteoarthritis, with supplements and creams (23.8%) also well represented. Exercise (2.9%) was the least recommended treatment therapy. General search engines returned 13.7% relevant websites while medical and meta-search engines returned 12.8% and 12.5% respectively (Table 1). There was no statistical significance in the efficiency of medical, general or meta-search engines. Of the individual search engines, AlltheWeb, a general search engine, was the most efficient, generating 36.4% relevant websites. Conversely NHSdirect, a medical search engine, was the least efficient, generating 6.3% relevant websites. Vivisimo was the most efficient meta-search engine, returning 29.2% relevant websites.

The overall quality of website information was poor, with the mean DISCERN quality rating score being 1.70 ± 0.67. Only 3% of websites rated above 3 on the DISCERN scale (i.e. provided potentially important information but with no serious shortcomings). The quality of websites sourced by medical search engines was of a significantly higher quality than those sourced by general or meta-search engines (P = 0.0001). The HealthInsite medical search engine sourced websites of the highest quality. The quality of information was significantly lower in websites that provided a direct service, such as selling of an arthritis-related product. Conversely, websites that demonstrated currency of information, had links to other sources, cited evidence, recognized authorship, had external source approval or recorded updating were more likely to have a significantly higher quality of information (Table 2). Higher-quality information was also associated with websites that provided information to both consumers and health-care professionals.


    Discussion
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
The results from this study demonstrate no difference in efficiency between general, medical and meta-search engines in sourcing osteoarthritis information on the Internet. Overall, the search specificity was low, with search engines only able to retrieve a small proportion (10%) of websites that were unique and offered relevant information about osteoarthritis. The overall quality of websites was poor; however, medical search engines identified higher-quality websites.

Previous studies examining search engine efficiency support our findings that medical search engines are no more efficient than general or meta-search engines in sourcing relevant health information [11, 12]. These studies, which searched for information on androgen deficiency in the ageing male (ADAM) and prostate cancer screening, are in direct contrast to the findings of this study with respect to the value of medical search engines in identifying higher-quality information. This difference in the quality of websites retrieved by medical search engines may be related to the lack of a definitive evidence base regarding ADAM and prostate cancer screening. For topics where evidence is clear, such as osteoarthritis, medical search engines appear to be meeting their goal of filtering the Internet to retrieve less, but more valid, information. However, in conditions that lack a definitive evidence base search engines appear less likely to retrieve relevant websites offering high-quality information.

While medical search engines may in some cases be helping, the Internet is currently falling short of its potential to facilitate evidence-based decision-making. Even in the presence of the strong evidence base for osteoarthritis, online information is rarely supported by evidence. In this study fewer than a third of relevant websites cited evidence to support their recommendations. Many websites made claims unsupported by the literature, whilst others, whose recommendations concurred with the evidence, made no reference to existing literature.

Consumers are generally not aware of characteristics that indicate quality information on the Internet [14]. Education about quality indicators such as (i) a clear statement of the purpose and target audience for the website, (ii) core standards for information provision such as authorship, disclosure, attribution and currency, (iii) citation of the best available evidence, (iv) acknowledgement of risks, other treatments and the effect of no treatment, (v) references to further sources and (vi) clear disclosure of sponsorship, affiliations or conflicts of interest, has the potential to increase consumer ability to access and resource quality information, providing a means of consumer empowerment and up-skilling. Bringing governance and editorial control to ensure the validity of Internet-based health information is an impossible task. Empowering and educating consumers to find and recognize valid health information may therefore be the only way for the Internet to reach its potential to inform, rather than mislead, health-care decisions.

This study assessed the nature, accessibility and quality of website information about osteoarthritis. Due to the dynamic nature of the Internet and the variation between the evidence base of different clinical conditions, the findings of this study may not be generalizable to other clinical areas. Future research could be directed towards investigating the value of medical search engines for other common conditions to determine if the results are consistent across other conditions with a strong evidence base.


    Acknowledgments
 
This research was funded by the National Arthritis and Musculoskeletal Conditions Improvement Grants Program, from the Australian Department of Health and Ageing.

The authors have declared no conflicts of interest.


    References
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 

  1. Towheed T, Judd M, Hochberg M, Wells G. Acetaminophen for osteoarthritis. In: The Cochrane Library, Issue 3. Chichester: John Wiley, 2004.
  2. March L, Bagga H. Epidemiology of osteoarthritis in Australia. Med J Aust 2004;180:S1–S6.
  3. World Internet usage statistics and population statistics. Available at http://www.internetworldstats.com (accessed 24 April 2004).
  4. Brown M. Consumer health and medical information on the Internet; supply and demand. Available at http://etrg.findsvp.com/health/mktginfo.htm (accessed 24 April 2004).
  5. American Medical Association. Study on physicians’ use of the World Wide Web. Chicago, IL: AMA Press, 2002.
  6. Aguillo I. A new generation of tools for search, recovery and quality evaluation of world wide web medical resources. J Manage Med 2000;14:240–8.[CrossRef]
  7. Charnock D. The DISCERN Handbook. Abington, Oxford: Radcliffe Medical Press, 1998.
  8. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Commun Health 1999;53:105–11.[Abstract]
  9. DISCERN online. Available at http://www.discern.org.uk (accessed 24 April 2004).
  10. Sacchetti P, Zvara P, Plante, P. The Internet and patient education—resources and their reliability: focus on a select urologic topic. Urology 1999;53:1117–20.[CrossRef][ISI][Medline]
  11. Ilic D, Bessell T, Silagy C, Green S. Specialised medical search engines are no better than general engines in sourcing consumer information about androgen deficiency. Human Reprod 2003;18:557–61.[Abstract/Free Full Text]
  12. Ilic D, Risbridger G, Green S. Searching the Internet for information on prostate cancer screening: an assessment of quality. Urology 2004;64:112–16.[CrossRef][ISI][Medline]
  13. NHMRC. How to use the evidence: assessment and application of scientific evidence. Canberra: National Health and Medical Research Council, 2000.
  14. Eysenbach G. Kohler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests and in-depth interviews. Br Med J 2002;324:573–7.[Abstract/Free Full Text]
Submitted 18 August 2004; revised version accepted 29 October 2004.



This Article
Abstract
Full Text (PDF)
All Versions of this Article:
44/3/382    most recent
keh498v1
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Search for citing articles in:
ISI Web of Science (3)
Disclaimer
Request Permissions
Google Scholar
Articles by Maloney, S.
Articles by Green, S.
PubMed
PubMed Citation
Articles by Maloney, S.
Articles by Green, S.
Related Collections
Other Rheumatology