Department of Rheumatology, Division of Immunity and Infection and
1 CRC Clinical Trials Unit, University of Birmingham, Edgbaston, Birmingham B15 2TT and
2 Department of Rheumatology, Dudley Group of Hospitals NHS Trust, The Guest Hospital, Tipton Road, Dudley, West Midlands DY1 4SE, UK
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods. Out-patients (20 with systemic lupus erythematosus and 31 with vasculitis) were randomly selected to complete either paper (n=29) or electronic and paper SF-36 versions (n=51) before and after consultation (paper vs paper comparison). Data were evaluated as the response correlation, internal consistency, missing data, patient satisfaction and preference.
Results. There were very good correlations in SF-36 responses (P<0.001) between the paper and electronic forms and the paper and paper forms. Internal reliability coefficients (Cronbach's ) showed good internal consistency for all reported responses in either computer or paper forms. There were no missing data in the computerized version but 24% of patients failed to answer all of the paper form questions. Ease of use of the computer version was rated highly by 71% of all the respondents, and 69% would prefer to use the computer version in future.
Discussion. Computerized data collection is acceptable to patients and feasible in clinical settings. It provides responses that are at least comparable to those to the paper form, improves data capture and is available immediately.
KEY WORDS: Rheumatology, Systemic lupus erythematosus, Vasculitis, SF-36, Electronic data capture, Quality of life.
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
We have routinely collected outcome data on patients visiting our clinics. This has entailed using indices such as the Vasculitis Integrated Total Assessment Log (VITAL) [7] and the British Isles Lupus Assessment Group (BILAG) index [8] and the Systemic Lupus International Collaborating Clinics/American College of Rheumatology (SLICC/ACR) damage index [9]. An important component of these indices is the evaluation of the patient's own perception of their health. This has been accomplished using instruments such as the Medical Outcome Studies Short Form (SF-36) [10], a self-assessed multilevel utility for the evaluation of both the physical and mental well-being of a patient [10]. It comprises eight major facets (Physical Function, Role Physical, Bodily Pain, General Health, Vitality, Social Function, Role Emotional and Mental Health) [10].
Assessments such as these are complex and time-consuming and require data manipulation before they can be used for clinical decision-making. A recent audit of SF-36 data collection on paper forms within our clinics over a 6-yr period showed that 154 patients who were habituated to this had attempted 436 consecutive SF-36s but only 336 (77%) had actually been completed [11]. This was despite repeated explanation of how to fill in the form, why it was important to complete all the questions and why the data were being collected.
Successful evaluation of disease status within a clinic relies upon efficient information collection and management and the immediate availability of results. With continual improvement in technology, the development and implementation of electronic information systems aimed at addressing clinical problems is desirable and is becoming more common [1214].
Over the last 2 yr we have been developing clinical electronic databases designed to aid data collection for use in various specialist rheumatology clinics. Within these systems, we have incorporated an electronic version of the SF-36. This was designed in order to reduce the time taken for the information to pass from the patient to the consulting physician and, hopefully, to increase the compliance in answering the questions posed in the questionnaire. Several studies have already shown that other health assessment questionnaires, used to collect quality of life information from cancer patients, have benefited from computerization [15, 16].
In the present study we evaluated, comparatively, the original paper methods and a newly developed, simple electronic system for SF-36 data capture. We assessed (i) reproducibility by comparing responses to the electronic questionnaire with those given in the paper format; (ii) effectiveness and efficiency by assessing the number of unanswered questions to the individual SF-36 questions, and (iii) acceptability of and preferences for the two systems by asking the patients to complete a preference questionnaire after completion of the two styles of SF-36.
![]() |
Patients and methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Patients
To assess the reproducibilities of the computer and paper versions of the SF-36, over a 6-month period 51 patients [31 vasculitis and 20 SLE patients of mean (S.D.) age 50 (14.7) and 43 (12.0) yr respectively] were selected from alternating positions of each clinic list. This allowed sufficient time for all the necessary explanations about the study and the system, and for an auditor to record impartially all the measures described, after the patients arrival and before their departure from the clinic.
Patients were informed of our intention to conduct the study and what it entailed, including the need to use a computer. Participants were not selected on the basis of their ability to use such technology. All patients invited to participate agreed to do so. Each patient was asked either to complete the computerized version before and the paper version after their clinical consultation or vice versa. Neither medium was promoted over the other.
To control whether the before/after approach affected the patients responses, 29 patients (all with vasculitis, seven of whom had previously participated in the computer vs paper evaluation) were asked to complete paper SF-36 forms both before and after their clinical consultation.
The system
The electronic interface was designed using a simple computer interface of an Access 97 form (Fig. 1) programmed with Visual Basic for Applications (Microsoft), the data being collected in underlying tables. The questions posed on the computer screen were phrased exactly the same as those in the paper questionnaire and were presented in the same style (with the same words underlined and the same punctuation). Patients were asked to use a standard mouse as a means of navigating through the menus and questions.
|
As this study involved a change to the normal routine of SF-36 data collection in our clinics, we used unique identifiers for the patient. Each patient was assessed individually in a private room off the main patient reception area, thus maintaining patient confidentiality throughout the study and also reducing disruption to the normal functions of the clinic.
Analysis
Reproducibility.
We compared the answers reported on the paper form vs the electronic version using Pearson's correlation coefficient. The internal reliability of the SF-36 scale scores was measured using Cronbach's . Where this was not appropriate (i.e. for facets comprising the two items Bodily Pain and Social Function), Spearman's
correlation coefficient was calculated.
Compliance.
As unanswered questions are one problem encountered when using the SF-36, we recorded the total number of missing answers on both the paper and the electronic form. SF-36 composite scores were not corrected for missing data; if data were not present, the score for that facet was not calculated and was omitted from the analyses.
Patient's preference.
Patient's preferences for the two versions of SF-36 were recorded. This included their opinions on the ease of use and clarity of the layout, both of which were determined using a five-point scale score (1=excellent, 5=very poor). The results were analysed with the MannWhitney U-test. We also asked the patients which system they liked using and which they would like to see adopted for future use.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
There was very little difference between the responses given to any of the facets of the SF-36 by all patients when they used either the electronic system or the standard paper questionnaire (Table 1). Correlations ranged from r=0.80 (Social Function) to r=0.96 (Physical Function) for the whole patient group. These were all highly significant (P<0.001).
|
Internal consistency coefficients (Cronbach's and Spearman's
) showed that all reported responses had good to excellent internal consistency for both the computer and the paper form, which were very similar to those previously reported for each SF-36 dimension. Cronbach's
ranged from 0.83 to 0.94 (paper version) and from 0.84 to 0.95 (computer version). Spearman's
correlation coefficients ranged from 0.68 to 0.97. All were highly significant (P<0.01).
There was no difference in the degree of concordance between the reproducibility of responses given to the electronic and paper versions of the SF-36 when we analysed the responses as two groups (younger than the mean age of 47 yr and older than this age).
Compliance
A review of the completeness of the SF-36 questionnaires showed that there was 100% compliance using the electronic version of the SF-36. This was not the case with the paper questionnaire: 24% of responders [10% in SLE clinic, 36% in the vasculitis clinic; 2(1)=4.16, P<0.05] failed to answer all of the SF-36 questions.
Patient preference
Of the 51 patients tested on the computer, all were very keen to complete the task; no-one refused to finish the computer test. Sixty-seven per cent of patients preferred the computerized version to the paper version (SLE group 71%, vasculitis group 64%) (significantly more than the paper version; P<0.01). When asked their preferences for adoption of these systems in future, 69% of patients requested the computerized version (SLE group 78%, vasculitis group 63%) (again significantly more than the paper version; P<0.01).
The 51 patients who completed the preference questionnaire were classed into two age groups about the mean (47 yr). Twenty one of these (42%) were younger and 29 (58%) were older. When we asked users their preference of medium, we found that significantly more of the younger group preferred the computer (n=17, 81%) when compared with the older age group (n=12, 41%; 2(1)=7.83, P<0.01). There was a similar response when the patients were asked their preference for future use of these systems.
When patients were asked to give a general opinion of why they preferred a particular system, ease of use and speed (71%) were given as reasons why the computer version was favoured. The patients felt that the on-line SF-36 took less time to complete; this was a subjective response as they did not time the exercise themselves. However, a review did indicate that average time to complete the computer form was 5 min, ranging from 4 to 10 min, whereas the patients could take a lot longer to answer the paper forms as they were habituated to answering the paper forms at their own pace whilst they waited for their consultation. Lack of computer literacy (54%) was the main reason given why people did not like using the computerized version. When all patients were asked to rate the individual systems in terms of their ease of use and clarity on a scale score, the computerized system was rated better than the paper version (Table 2).
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Gathering data from patients within busy real-life working specialist rheumatology clinics is hindered by the practicalities of their day-to-day running. Patients have been habituated to completing the SF-36 health assessment questionnaire whilst they attend the clinics, but they have generally been left to complete the SF-36 in their own time and at their own pace. Collecting the data on paper requires further transcription to computer before the results are available for the clinician; this precludes review of data completion, leading to the loss of data.
With the burgeoning developments in information technology, simple, effective yet highly specialized electronic data capture systems can be easily developed and implemented with the intention of improving practices [21, 22]. We have developed a system which allows effective collection of SF-36 data from patients with little or no intervention by the clinical or nursing staff. This circumvents the need to transpose the information from the paper questionnaires which the patient has previously completed. This facilitates the information-gathering process and reduces the delay in monitoring the changes in patient responses, and the information is available to the clinician immediately.
Improvements in the mechanisms for data collection need to be assessed prior to adoption. We have developed a simple electronic graphical user interface for the SF-36, prepared in line with guidelines for such design [23, 24]. Although this was an initial version tested for usability, we tried to incorporate into the design aspects which we thought would be helpful to patients hindered by the limitations of conditions such as SLE and vasculitis. This included large, easy-to-read text, easy navigable interfaces and large, simple-to-use buttons for them to record their responses.
To test this system, we invited attendees of two of our specialist rheumatology clinics to try the computer programme. These clinics were chosen in order to evaluate how this system would operate in two distinctly different styles of clinic.
Although it might be expected that a change in the style of presentation of the questionnaire (electronic as opposed to paper) could influence the response, this did not seem to be the case. We did not find that there was any significant difference between the responses given by the patient to any of the major components of the SF-36, whichever medium they were asked to complete. Evaluation of the internal reproducibility of each method (Cronbach's ) showed that, irrespective of the medium used, patients were fairly consistent in their responses to each question and the medium used did not affect their reported scale scores for the SF-36.
The cross-over study design attempted to control for any effects that completing the forms before and after consultation or the stay in the clinic may have had upon the responses given. We did not want to expose the patient to either style of SF-36 immediately after the other in order to prevent them from duplicating their answers from one medium to the other. Therefore, we imposed an intervening period during which the patient would visit the clinician. This is not an ideal situation, as consultation may have influenced the SF-36 results. We tried to minimize this effect by alternating the medium that was attempted first and by adopting as the third arm of the study the paper vs paper evaluation: before and after consultation. This latter arm assessed if length of stay in the clinic/contact with the physician affected the patients responses to the SF-36 questions. We did not find this to be the case and therefore deemed that it was not necessary to pursue this aspect any further.
The introduction of the computer format into our clinics resulted in 100% compliance with completion of all the questions of the SF-36 when compared with the paper collection aspect of the study. These findings are similar to those of Velikova et al. [16], who reported that their patient cohort achieved full compliance in their answers to their electronic quality-of-life questionnaire. Comparisons of paper (mailshot) and telephone administration have shown that mailed SF-36s are more likely to be incomplete when compared with a telephone survey, with about 3:1 unanswered questions [25]. This could indicate that some degree of personal contact may aid compliance to answering the questionnaires and that self-assessment paper forms are not the most advantageous method of collecting data.
The electronic system was well accepted by the patients, a majority favouring the computerized form over paper. The main reason for this was that they believed the former could be completed more quickly. The novelty of the computer system and the presence of the auditor conducting the study may have made it appear to be quicker to fill in the computerized form; however, patients had normally been used to filling in the paper forms at their convenience. This could take anywhere between 5 min and half an hour, depending upon the patient. The distinct advantage of the computer system in terms of speed is the calculation and presentation of the data. Paper forms have to be transposed into relevant computer applications in order to calculate the SF-36 components, and this adds considerably to the time taken to produce a result.
The lack of computer literacy was the main reason why people were initially hesitant about an electronic style of questionnaire. This is consistent with findings reported from other studies which had evaluated the acceptability of computerized versions of other health-related quality-of-life questionnaires [15]. We did find an age difference in preference for the type of system. Splitting the patients into nearly equal groups around the mean age (47 yr) showed that the younger patients preferred the computerized system whereas the older ones still preferred to use paper forms. The slightly lower mean age of the SLE patients could explain their greater preference for using a computerized system. Despite this, the patient's age had no bearing on the reproducibility of the responses given to either the paper form or the electronic version.
With better design of the computerized SF-36, it may be possible to introduce a system which is acceptable to all patients. Reports have already shown that quality-of-life questionnaires using touch-screen technology are an acceptable approach for presenting information [15, 26, 27]. In the present study, patients used a standard mouse for navigation through our interface. However, as rheumatic diseases can manifest themselves in debilitating joint mobility problems it may be possible to enhance the acceptance of this method of capturing data by using touch-screens.
Disease activity scores and patient global assessments are now being accepted as viable tools in clinical research [28]. The development of effective methods of recording this information, as we have with our electronic system, will improve the quality and quantity of data that we collect. Use of computers in a clinical setting is now being seen to be an acceptable activity, patients regarding this as normal and indicative of doctors being up to date [29]. There is little evidence to suggest that this can lead to loss of the personal touch as long as verbal skills and eye contact are maintained [29].
In order to evaluate whether a computerized SF-36 could be used in larger, more generalized rheumatology clinics, it would be necessary to perform a more in-depth assessment. We have not attempted to do that here but have described one method that has been effective in achieving our goals of collecting quality research data in a busy working clinic. This system has been acceptable to our patients and has allowed the clinician to have a complete patient record, with immediate feedback about the patient's progress over time.
![]() |
Acknowledgments |
---|
![]() |
Notes |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|