Development, validation and use of a patient knowledge questionnaire (PKQ) for patients with early rheumatoid arthritis

S. L. Hennell, C. Brownsell1 and J. K. Dawson1

Department of Rheumatology, Wirral Hospital NHS Trust, Wirral, Merseyside and 1Department of Rheumatology, St Helens and Knowsley Hospitals NHS Trust, Merseyside, UK

Correspondence to: S. Hennell. Rheumatology Department, Arrowe Park Hospital, Arrowe Park Road, Upton, Wirral, CH49 5PE UK. E-mail: sheena.hennell{at}whnt.nhs.uk


    Abstract
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 Conclusion
 References
 
Objectives. To design and validate a patient knowledge questionnaire (PKQ) to use pre- and post-education programme for a group of early rheumatoid arthritis (RA) patients. The aim was to design a tool to assess knowledge acquisition before and after an arthritis information course (AIC).

Methods. The PKQ design incorporated 12 multichoice questions. Validation procedures were undertaken. Following PKQ validation 30 patients were recruited from specialist practice having met the inclusion criteria. All patients attended the arthritis information course run by the multidisciplinary team. Demographic data such as age, gender, disease duration and educational level were evaluated for possible significance.

Results. The PKQ demonstrated test–retest stability with a Pearson's correlation coefficient of 0.965 and an intraclass single measure coefficient of 0.87. Internal consistency was demonstrated with a Cronbach's alpha of 0.62. The PKQ was sensitive to change with a statistically significant improvement following three 1-h education sessions (Mann–Whitney U-test P<0.001). No correlation was found between baseline PKQ scores and age or disease duration. However, significant correlation between educational level and pre-knowledge scores (Pearson's correlation r = 0.386, P = 0.035) was demonstrated. Change in PKQ score was not associated with age, disease duration or educational level. Patients with lower initial pre-scores obtained the most improvement post-AIC ({chi}2-test P = 0.003).

Conclusions. The PKQ is a reliable, valid and sensitive instrument suitable for measuring the acquisition of RA knowledge in a group of early RA patients following arthritis information courses.

KEY WORDS: Patient knowledge, Questionnaire, Rheumatoid arthritis.


    Introduction
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 Conclusion
 References
 
Rheumatoid arthritis (RA) is a chronic, painful, disabling disease. Prognosis is uncertain and treatment is often complicated involving long-term medication, maintaining energy conservation and preventing joint deformity [1]. The disease is often characterized by flare-ups and patients therefore require regular review and ongoing support and information during the course of their disease.

There is also growing evidence that many patients experience significant disability in the very early stages of RA [2] owing to the often very variable nature of the disease. At this stage health professionals can play a key role in reducing the effect of the disease through assessment, understanding need and providing meaningful information [3]. However, debate tends to centre on the timing of such interventions.

Current opinion has yet to agree on the optimal time for presenting patients with disease-related information [4]. It could be suggested that early onset of RA is the optimum time to assess the patient's individual educational need. This would enable information provision, support and the answering of questions to begin to empower patients to self-manage. This view is shared by others who state that, from diagnosis onwards, those with RA require the necessary knowledge and understanding to enable them to make informed decisions regarding treatment options and to carry out self-care [5]. This suggests that education programmes should be targeted at those with early RA. In addition, when questioned, those with RA that had been diagnosed for less than 5 yr felt patient education aided their understanding of RA disease compared with those with more long-standing disease [6]. Current evidence also suggests that there is the potential for greater reduction in disability with early educational intervention [7].

It is vitally important therefore that nurses are aware and can demonstrate the effectiveness of education programmes. One way to demonstrate this effectiveness is to measure patient knowledge following participation at an education programme. If post-intervention knowledge scores are higher, this supports the evidence of effective patient education.

Two validated patient knowledge questionnaires—the Hill PKQ [8] and the ACREU (Arthritis Community Research and Evaluation Unit) [9]—are currently available to measure RA knowledge. The Hill PKQ appears complicated to follow in terms of structure and the responses require patients to use different numbers of true statements to answer each question. The questions include medical terminology and are quite lengthy. The low initial scores provide evidence that the questionnaire appears difficult to follow as, despite mean disease duration of 9.6 yr and high level of education, scores were low [8]. The ACREU [9] questionnaire is based on a 5-point Likert scale. This style is not suitable for a questionnaire, which is based on factual questions and measuring knowledge [10]. The authors themselves admit this questionnaire is probably not relevant to other teaching programmes and it has not been tested with patients who are newly diagnosed. However, significant increases in knowledge scores (P<0.01) are demonstrated between the education and control groups at 6 weeks [9]. These tools, although valid and demonstrating results, have limitations in practice. Therefore, in order to demonstrate knowledge change pre- and post-education programme, the decision was taken to design a PKQ.

The primary aim of this study was therefore to design and validate a PKQ, for use with early RA patients, and to investigate whether it is sensitive to change following a departmental arthritis information course (AIC). A secondary analysis compares PKQ scores with patient demographics and investigates retention of knowledge.

The arthritis information course (AIC) was designed to provide those patients identified as having early RA with the necessary information/knowledge to self-manage this chronic disease effectively. The key component of this education programme is that the content is based on the needs of those receiving the education and therefore the Multi Disciplinary Team (MDT) using specific topic areas (see Table 1) designed the course. The teaching component covers two sessions attended by 8 to 10 patients following initial baseline assessment, aimed at identifying individual educational need.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Topic areas for arthritis information course (AIC)

 

    Patients and methods
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 Conclusion
 References
 
Design of the patient knowledge questionnaire (PKQ)
The PKQ was developed and comprised 12 multichoice questions with a choice of five responses. Specific areas from the AIC were identified for testing in the questionnaire. These were aetiology, signs and symptoms, drug therapy and monitoring, joint protection and exercise and energy conservation. The questions were written without jargon and difficult terminology to aid understanding, using the Devaus framework [11] and some sample questions are included in Table 2. Various ways of wording questions were considered to avoid the possibility that certain responses may be consistently chosen in error. This was important, as a particular phrasing of a question may ultimately be misleading to the respondent.


View this table:
[in this window]
[in a new window]
 
TABLE 2. Sample questions from PKQ

 
Demographic data including gender, age, disease duration (<2 yr) and educational ability were collected on the front sheet of the PKQ. This enabled the researcher to bracket the educational levels of the sample. Education level 1 comprised those below A-level standard and education level 2 those with A levels (or equivalent) and above.

An independent rheumatology panel comprising senior physiotherapist, occupational therapist, a rheumatology nurse practitioner and rheumatology consultant reviewed the questions and confirmed them to be representative of the AIC content, in order to confirm the content validity of the PKQ.

The pilot study was necessary so that a preliminary test of the questionnaire could be conducted. Any problems/benefits associated with the design could then be identified. The first draft was prepared following consultation with the multidisciplinary team. Following this the questionnaire was piloted (for completion of the PKQ) to the 15 patients and other members of the MDT, not included in the development, for comments on the question content and design. Analysis of the completed questionnaires identified some problems. These related to format, some question phrasing, some answer/response phrasing and relevance and suitability of possible answers and are highlighted with changes in Table 3.


View this table:
[in this window]
[in a new window]
 
TABLE 3. Wording problems/errors in the PKQ

 
During the initial pilot and pre-testing the demographic question relating to level of education drew some criticism. Patients did not want to feel ‘educationally graded’ prior to completing the PKQ. Therefore, asking the patient to provide the information seemed the right way to proceed. The following was devised, adapted from an original tool [12]: (1) At what age did you leave school?; (2) Have you studied since leaving school?; and (3) If so, what qualifications have you gained?

Following re-pilot this was found to be much more acceptable to the patients who felt that by giving this information they were now personally involved in the study. This scale still enabled the researcher to bracket the educational levels of the sample to determine any possible correlations within the data. The pilot (allowing for changes) provided evidence that the PKQ questions worked as intended and that the use of editing helped to reduce the potential for problems.

Patient selection
Patients were therefore recruited to the main study having met the following inclusion criteria: newly diagnosed with RA for <2 yr, no previous attendance at an RA education programme and maximum age of 75 yr. Patients who were deaf or unable to read were excluded. Written consent was obtained from all patients after perusal of the information sheet. Ethical approval for this study was granted by St Helens and Knowsley Local Research Ethics Committee.

Convenience sampling was used to recruit early RA patients, those newly diagnosed with RA and referred to the AIC. Patients were consecutively recruited to participate in the pilot study and test–retest analysis and were grouped as follows: (1) 15 RA patients (2–5 yr after diagnosis) for the pilot study to comment on question format, with some knowledge but not extensive knowledge, to test PKQ for sensitivity to change and to provide data for power calculation; (2) a further 40 RA patients recruited from weekly monitoring sessions to answer PKQ for test–retest analysis (in treatment room) 1 week apart; and (3) 30 newly diagnosed RA patients to participate in study (29 patients was the minimum number resulting from the power calculation required to show the effect of the educational programme on patient knowledge).

The data collection process was conducted by two nurse specialists who administered the 30 questionnaires pre- and post-AIC and at 3 months, obtained consent, collected questionnaires, and both marked and second marked and confirmed data entries. Data from the questionnaires was entered into a spreadsheet using SPSS data entry.

Statistics
Test–retest data were used to assess the stability and internal consistency of the PKQ. Intraclass correlation coefficient analysis was applied to assess stability using a 2-way mixed effect model (consistency definition) to provide an alpha coefficient for each individual's test and retest response. Overall score stability was assessed using Pearson's correlation coefficient. The method used for assessing internal consistency of the baseline scores was Cronbach's alpha. The minimum acceptable level for internal consistency coefficient is between 0.50 and 0.70 [10].

Continuous data are presented through descriptive statistics, i.e. mean and standard deviation (S.D.). Correlation between pre-education scores and post-education scores and demographic data were compared using Pearson's correlation coefficient. Paired t-test was used to compare normally distributed data. Mann–Whitney U-tests were used to compare data where normal distribution could not be assumed. {chi}2-test was used to compare frequencies. For 2 x 2 tabulations of frequencies when expected counts were below 5, Fisher's exact test was used. The above were all calculated using the statistical package SPSS for Windows release (11.0.0).

During the design stage, the sample size was calculated via a power calculation (80% at the 0.05 level of significance) using the Minitab version 13.20 statistical software package. From the pilot study of 15 patients, mean change in score = 1.2, standard deviation = 1.6, effect size = 0.75. A sample size of at least 29 patients was required for this study.


    Results
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 Conclusion
 References
 
Reliability, stability and internal consistency
The analysis of the internal test–retest data gave a single measure intraclass correlation coefficient of 0.87 with 95% confidence interval of 0.85–0.89. The test–retest overall score gave a Pearson's correlation coefficient of 0.965, which is significant at the 0.01 level (2-tailed). Internal consistency was demonstrated with a Cronbach's alpha of 0.62. These results support the reliability and high stability of the PKQ to testing.

Validity
Subjectively the PKQ demonstrates face and content validity.

PKQ results pre-and post-AIC
Thirty patients participated in the study. All 30 patients completed pre- and post-PKQs, but four patients failed to complete the 3-month PKQ. Demographic data are shown in Table 4, and Table 5 shows pre-AIC scores and characteristics. Mean scores from the study (n = 30) show men achieving a mean pre-AIC score of 7.6 and women 7.9. The post-AIC score increased to 9.2 for men and 9.9 for women. At 3 months the mean score for men was 8.4 and women 9.7.


View this table:
[in this window]
[in a new window]
 
TABLE 4. Baseline characteristics of the patients in main study (n = 30)

 

View this table:
[in this window]
[in a new window]
 
TABLE 5. Pre-AIC scores and characteristics

 
There was no correlation between age or disease duration with pre-AIC scores. However, there was a significant correlation between educational level and pre-AIC scores (Pearson's correlation r = 0.386, P = 0.035) (see Fig. 1). This shows that education level 1 correlates with lower pre-AIC scores and education level 2 correlates with higher pre-AIC scores.



View larger version (58K):
[in this window]
[in a new window]
 
FIG. 1. Relationship between educational level and pre-test score.

 
PKQ scores showed a significant increase post-AIC (Mann–Whitney U-test P<0.001 with 95% confidence interval of 1.2–2.54). PKQ scores at 3 months indicated a near significance (Mann–Whitney U-test P = 0.06 with 95% confidence interval of –0.05–2.13) towards knowledge being maintained at 3 months. The dropouts at this stage make the analysis underpowered and therefore probably affect the results. Immediate post-intervention scores showed a significant association between pre-education scores and improvement (Mann–Whitney U-test P = 0.003 with 95% confidence interval of 1.25–4.51), i.e. patients with the lowest scores improved most with AIC. However, improvement post-AIC showed no specific association between age, disease duration or education level and change in PKQ score.

No association was demonstrated between change in 3-month score and age, disease duration or education level.

Post-AIC there was no significant difference in the improvement between men and women. However at 3 months there is a statistically significant association ({chi}2-test P = 0.05) towards females maintaining knowledge better than men (shown in Table 6).


View this table:
[in this window]
[in a new window]
 
TABLE 6. Three-month post-AIC results according to sex of patient

 

    Discussion
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 Conclusion
 References
 
The PKQ was designed to test patient knowledge and has been shown to be reliable, valid and reproducible, through a process of appropriate testing. It has demonstrated a wide variation in scores, pre-AIC scores ranging from 2 to 11 and post-AIC scores from 6 to 12. Pre-intervention scores in this early RA group were low and did show a statistically significant increase post-intervention. Those who gained lower pre-intervention scores in either level 1 or 2 showed greater improvement than those with higher original pre-education scores. This PKQ has generated a good response rate from this early RA group over the study period. It is quick and easy to complete owing to the format chosen and appears very acceptable to patients.

This study confirms that patient knowledge increases following group educational intervention, as previously shown [13, 14]. However this has not before been demonstrated in those with early RA. As discussed, timing of educational interventions can alter outcome. The AIC has been specifically targeted at those patients with early RA resulting in significant knowledge gains. This is particularly relevant when considering the evidence that early educational intervention greatly reduces the chance of disability [7] and in addition knowledge gained early on in the disease process sets the trend for personal disease management, which continues throughout life [7].

A specific limitation of the study concerns the size of the sample. It is recognized that this study does have a small sample; however, the flow of the study and in particular the data collection coincides with the pattern of the AIC, and recruitment to this is dependent on newly diagnosed RA patients. The power calculation using the pilot study data required a sample of at least 29 patients in order to show a statistical significance and this was achieved as the sample included 30 patients. Importantly the sample was large enough to show that the PKQ is sensitive to change and is therefore extremely useful at recording increases in knowledge scores in the small groups of early RA patients attending education.

This PKQ is unable to show criterion validity, because as yet there is no recognized accepted gold standard to use for comparison. The Hill PKQ [8] is too difficult as demonstrated by low scores despite long-standing disease and the ACREU [9] Likert response is unsuitable for multichoice; in addition neither were designed or validated for use in early RA.

There is some interesting evidence relating to gender that emerged from this study. Initial improvement in the PKQ post-AIC was not significantly different between men and women. However at 3 months there is an association towards females maintaining PKQ scores better than men. Perhaps the education programme is too female orientated? Changes can be made to delivery of education and the PKQ can be used to assess whether these improvements have been effective. With larger numbers we may be able to identify patients who do not improve with group education and offer them individual education as an alternative.

In addition it is useful to know how much knowledge is retained post-AIC and this is currently evaluated at 3 months. However, further evaluation is required with more patient numbers to be able to show statistical evidence of retention, consistent with other findings where knowledge gain was maintained at 12 months and 5 yr, respectively [14, 15]. Thus far this PKQ has not been used in any other rheumatology unit. To ensure that these results can be generalized to the wider RA population this would need to be undertaken.

In addition, further recommended study would require correlation of knowledge scores with other outcome measures such as disease activity score (DAS), health assessment questionnaire (HAQ) and RA quality of life (QOL). This may facilitate further means with which to demonstrate effectiveness of patient education in those with early RA. The authors recognize that knowledge is not the only measurable effect of AIC and other meaningful effects are measured using the short form (SF)36 and patient satisfaction questionnaires.


    Conclusion
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 Conclusion
 References
 
This study is important for quality issues such as clinical governance, audit and benchmarking to move towards standardizing outcomes for educational programmes and allowing comparison of audit at a national level. Such findings are relevant to our practice and we must ensure that this impact is recognized as we move towards a National Service Framework for rheumatology.

The main advantage of this PKQ is that it has been designed specifically to assess changes in the knowledge scores of early RA patients attending a rheumatology unit. This PKQ is validated, sensitive to change and producing interesting results following AIC. Despite low education levels associated with low initial PKQ scores it appears that the AIC targeted at the correct level improves patient knowledge.

The authors have declared no conflicts of interest.


    Acknowledgments
 
Thank you to L. Woods (Liverpool University) and R. Wiswell (metrologist).


    References
 Top
 Abstract
 Introduction
 Patients and methods
 Results
 Discussion
 Conclusion
 References
 

  1. Snaith ML. ABC of rheumatology. London: BMJ Books, 1999.
  2. Sokka T, Kautiainen H, Mottonen T, Hannonen P. Work disability in RA 10 years after diagnosis. J Rheumatol 1999;26:1681–5.[ISI][Medline]
  3. Griffith J, Carr A. What is the impact of early rheumatoid arthritis on the individual? Best Pract Res Clin Rheumatol 2001;15:77–90.[CrossRef][ISI][Medline]
  4. Barlow J. Knowledge in patients with RA: A longer term follow-up of a RCT of patient education leaflets. Br J Rheumatol 1998;37:373–6.[CrossRef][ISI][Medline]
  5. Barlow JH, Cullen LA, Rowe IF. Comparison of knowledge and psychological well-being between patients with a short disease duration (<1 yr) and patients with more established RA (>10 yr duration). Patient Educ Couns 1999;38:195–203.[CrossRef][ISI][Medline]
  6. Kay E, Punchak S. Patient understanding of the causes and medical treatment of RA. Br J Rheumatol 1988;27:396–8.[ISI][Medline]
  7. DeVellis RF, Blalock SJ. Psychological and educational interventions to reduce arthritis disability. Baillieres Clin Rheumatol 1993;7:397–416.[ISI][Medline]
  8. Hill J, Bird HA, Hopkins R. The development and use of a patient knowledge questionnaire in rheumatoid arthritis. Br J Rheumatol 1991;30:45–9.[ISI][Medline]
  9. Lineker S, Badley E, Hughes E, Bell M. Development of an instrument to measure knowledge in individuals with RA: The ACREU RA knowledge questionnaire. J Rheumatol 1997;24: 647–53.[ISI][Medline]
  10. Bowling A. Research methods in health: investigating health and health services, 2nd edn. Buckingham: Open University Press, 2002.
  11. Devaus DA. Surveys in social research. Sydney: Allen & Unwin, 1985.
  12. Lubrano E, Helliwell P, Moreno P. The assessment of knowledge in ankylosing spondylitis patients by a self administered questionnaire. Br J Rheumatol 1998;37:437–41.[CrossRef][ISI][Medline]
  13. Davis P, Busch A, Lowe J, Taniguchi J, Djkowich B. Evaluation of a rheumatoid arthritis patient education program: impact on knowledge and self-efficacy. Patient Educ Couns 1994;24:55–61.[ISI][Medline]
  14. Lindroth Y, Brattstrom M, Bellman I, Ekestaf G. A problem-based education program for patients with RA: evaluation after three and twelve months. Am Coll Rheumatol 1997;10:325–32.
  15. Scholten C, Brodowicz T, Graninger W. Persistent functional and social benefit 5 years after a multi-disciplinary arthritis training program. Arch Phys Med Rehabil 1999;80:1282–7.[ISI][Medline]
Submitted 22 April 2003; Accepted 13 October 2003





This Article
Abstract
Full Text (PDF)
All Versions of this Article:
43/4/467    most recent
keh069v1
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Search for citing articles in:
ISI Web of Science (2)
Disclaimer
Request Permissions
Google Scholar
Articles by Hennell, S. L.
Articles by Dawson, J. K.
PubMed
PubMed Citation
Articles by Hennell, S. L.
Articles by Dawson, J. K.