1 Department of Pediatrics, Division of Rheumatology, Hospital for Sick Children, 2 Department of Pediatrics, Hospital for Sick Children, 3 Department of Health Policy Management and Evaluation, University of Toronto, 4 Graduate Department of Rehabilitation Science, University of Toronto and 5 Department of Public Health Sciences, University of Toronto, Toronto, Ontario, Canada.
Correspondence to: B. M. Feldman, Division of Rheumatology, The Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada. E-mail: Brian.Feldman{at}sickkids.ca
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods. We used a Delphi survey process in two steps. First we surveyed 187 paediatric rheumatologists and allied health professionals. We elicited 2607 items that, when combined with previously elicited items from parents/patients, could be pooled into 107 independent items. We then surveyed the paediatric rheumatologists to determine the frequency and importance of the 107 items.
Results. Our response rate was 83% to both surveys. We identified 29 items as being the most important and most frequently seen indicators of active disease. The most highly rated of these items were: presence of fever, presence of rash, elevated ESR, elevated CRP, requirement for increasing medications, abnormal physician global evaluation and presence of joints with active arthritis.
Conclusions. Twenty-nine items are thought by medical practitioners to be most relevant in determining disease activity in systemic arthritis. As a next step, the measurement properties of these items will be tested to help develop a disease activity tool.
KEY WORDS: Juvenile idiopathic arthritis, Disease activity, Measurement
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
There are currently no measures of disease activity available for SJIA. The lack of standardized validated measures of disease activity has significantly limited the ability to measure the impact of this disease and to determine its response to therapeutic interventions. This may be one of the reasons for the lack of well-designed trials for assessing the efficacy of different therapies in SJIA.
The concepts of disease activity, disease damage and disease severity have not been well defined in the context of JIA. Currently there is no consensus among paediatric rheumatologists regarding the exact meaning of these terms. A commonly used definition of disease activity is that component of severity that is reversible; activity may result in no or little damage in the future or be replaced completely by damage [4]. For our study, we defined disease activity as the reversible manifestations of SJIA.
Being able to quantify disease activity is important in the process of evaluating the efficacy of anti-inflammatory and remission-inducing treatments for arthritis; anti-arthritic treatments are aimed at reducing disease activity through their effects on the body's inflammatory pathways and immune system. Disease activity measures have already been developed for other adult and childhood rheumatic diseases, including dermatomyositis and systemic lupus erythematosus (SLE) [58].
Reducing disease activity is presumed to reduce future damage and therefore preserve quality of life. Indeed, the relationship between accumulated disease activity and disease damage has already been demonstrated in children with SLE [5, 6].
Our overall goal is to develop and validate a disease activity tool for SJIA. We plan to do this in several stages using a consensus approach. The tool is planned to include validated items that can be used to monitor disease and be used in future clinical trials. In contrast to other such attempts to develop disease activity tools, we decided to incorporate patient/parent perceptions of disease activity in the initial item generation phase of the process.
In our efforts to incorporate international opinion and consensus, we adopted the Delphi survey technique. The Delphi technique was developed by the Rand Corporation (and named after the famous oracle at Delphi). It has been widely used in health-care research [911]. According to Linstone, Delphi may be characterized as a method for structuring a group communication process so that the process is effective in allowing a group of individuals as a whole to deal with a complex problem [12].
The specific aims of this project were to identify candidate items and to achieve reduction of the identified items to a manageable list using international consensus through the Delphi survey technique. This is a first step in the development of an eventual disease activity tool.
![]() |
Materials and methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
Phase 1
Item elicitation survey design. For both phases of the study, item elicitation surveys were designed using Dillman's tailored design method [14]. In accordance with the principles enunciated by Dillman, the following steps were taken to improve the response rate: a respondent-friendly survey was designed with easy-to-understand language; four contacts were made by first-class mail and e-mail, with an additional special contact by telephone or fax; return envelopes with postage stamps were included in the survey mail-out; correspondence was personalized; the survey was printed on off-white paper to help the surveys stand out on a crowded desk; and a small gift was sent in appreciation of the respondent's effort (but was not meant to remunerate for time spent).
Prior to mailing, this survey was pretested for readability, face validity (sensibility) and comprehensiveness by administering it to five rheumatologists and two allied health professionals at The Hospital for Sick Children. An evaluation form asked the seven respondents to rate the survey for the clarity of language, the appropriateness of the phrasing of the questions, the ease of completion, and the suggestion of any bias on the part of the researchers. The respondents of the evaluation were also asked to list any other issues with the survey. The survey was modified based on their feedback and comments.
The survey was open-ended in design. Respondents were asked to list the variables that, in their perception, were reflective of disease activity in SJIA. Suggested headings, used as prompts, included clinical symptoms, clinical signs, laboratory tests and radiological findings. The respondents were also asked about how many individual patients with SJIA were seen in their centre every year.
The surveys were mailed initially. In order to increase response rate, a postal reminder was sent 2 weeks later, followed by mailing of the entire survey again in 4 weeks time. A personalized e-mail was sent at 6 weeks along with the survey attached to the e-mail. As a final step, the non-respondents were contacted personally by one of the investigators to solicit their participation.
Study respondents: physicians. Surveys were sent to paediatric rheumatologists internationally. In order to be objective and to avoid any potential bias, the heads of paediatric rheumatology organizations were asked to nominate individuals with expertise in SJIA; the physicians sampled were nominated by the heads of these organizations. The organizations were PRINTO (Pediatric Rheumatology International Trials Organization), PRCSG (Pediatric Rheumatology Collaborative Study Group), PRES (Pediatric Rheumatology European Society), BPRG (British Paediatric Rheumatology Group) and CPRA (Canadian Pediatric Rheumatology Association). Not all organizations were represented equally, as many respondents belonged to more than one of these groups. A total of 171 paediatric rheumatologists worldwide were sent the surveys.
Study respondents: allied health professionals. Sixteen allied health professionals were also sent the survey (therefore the total sample size for Phase I was 187). They were nominated by the head of the ARHP (Association of Rheumatology Health Professionals) for their expertise in paediatric rheumatic diseases.
As a separate study, parents and children with SJIA were interviewed for their input regarding disease activity. The parents and patients were chosen by purposive sampling to reflect the mixture of age, sex and disease severity seen with SJIA [13].
Phase 2
Item reduction. The 2607 items generated from the first phase were reduced to 107 by combining like items. This process was done by the investigators. All items elicited from the family interviews were included at this stage. Three of the investigators (A.V.R., R.S., B.M.F.) initially organized the 2607 items independently. They then met on two occasions to collectively compare the categories created and to establish consensus where there were differences in opinion about the individual categories. All the final items were decided based on complete consensus of the three investigators.
A second questionnaire containing these 107 items was designed asking respondents to rank each item for its importance and frequency on a five-point categorical scale. The questionnaire was again designed using Dillman's tailored design method. The respondents were also given a column alongside each item to list any comments they may have had on that item. At the end of the questionnaire the respondents were asked to list any items that may have been missed out from the initial phases that were considered by them to be important.
Prior to mailing, this second questionnaire was pretested for readability, face validity (sensibility) and comprehensiveness by administering it to five rheumatologists at The Hospital for Sick Children. The questionnaire was modified, based on their feedback and comments in an evaluation form, using a process similar to the first survey development.
The questionnaires were sent to the respondents by both e-mail and by post initially. In order to increase response rate, a postal reminder was sent 2 weeks later, followed by mailing of the entire questionnaire again in 4 weeks time. A personalized e-mail was sent at 6 weeks along with the questionnaire attached to the e-mail.
Physicians. One hundred and sixty-nine physicians who participated in the initial phase were sent the Phase 2 questionnaire (two of the original participants declined to participate in the future phases).
Data entry. Double data entry of all questionnaires was done to ensure accuracy of data.
The project was approved by the Research Ethics Board of The Hospital for Sick Children.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
The items that scored a maximum of 5 on the importance scale (i.e. items thought to be the most important indicators of active SJIA) were fever, rash, increased erythrocyte sedimentation rate (ESR), increased C-reactive protein (CRP), requirement for increasing medications, inability to taper off medications, abnormal physician global evaluation of disease activity, and joints with active arthritis.
The items that were thought to be seen most often as indicators of active SJIA (i.e. the most frequent items) were fever, rash, increased ESR, increased CRP, requirement for increasing medications, abnormal physician global evaluation disease activity, abnormal parent evaluation of disease activity, joints with active arthritis, morning stiffness, joint pain, fatigue, thrombocytosis, leucocytosis and anaemia.
For the 107 items, the median score of 14 items was 5 on the importance scale (most important), 33 items scored 4, one item scored 3.5, 35 items scored 3, one item scored 2.5, 18 items scored 2 and five items scored 1 (least important). On the frequency rating of the 107 items, the median score of 14 items was 5 (most frequently seen), 26 items scored 4, 28 items scored 3, 26 items scored 2 and 13 items scored 1 (least frequently seen) (see Appendix 1 for the list of 107 items and the median importance and median frequency).
The top 29 items from this phase are listed in Table 2. All the top 29 items had a median score of 4 or more on both the importance and frequency ratings (where 1 indicates little importance and 5 is very important; 1 indicates very infrequent and 5 very frequent). These 29 items will form the basis of a tool to be validated in future studies in patients.
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The items found to be important and frequently seen in SJIA fit naturally into domains: systemic symptoms (fever, rash, splenomegaly, lymphadenopathy), inflammatory markers (raised ESR and CRP), joint manifestations (joints with active arthritis, swollen joint count, morning stiffness, joint pain), functional measures, quality of life (QOL) measures, physician's global and parent's/patient's global assessment of disease activity.
Items that were thought to be significant but not found in the eventual list include pericarditis, myocarditis, Koebner phenomenon, weight loss, sore throat and features of macrophage activation syndrome. Some of these, such as pericarditis and myocarditis, were thought to be important but not frequently seen. Some items (weight loss, fatigue) were thought to be frequently seen but not important (perhaps because of poor specificity) as indicators of active disease. Overall, we feel our respondents understood the purpose of the exercise as being to come up with a core selection of items for a disease activity tool that could be easily and widely used.
We used the Delphi process in our attempt to develop a disease activity tool. The Delphi technique enabled us to elicit opinions (items) from a diverse group of paediatric rheumatologists in an anonymous fashion. This process allowed us to get high-quality ideas from an international group of participants; this should allow the eventual tool to have broad acceptance.
In order to ensure patient/parent input, the initial phase included items generated from an interview with 14 patients/parents chosen by purposive sampling to be representative of age, gender, disease activity and disease severity. We feel that our method of including allied health professionals and patients/parents in the initial phases broadened the perspective and allowed us to include all potentially important indicators of disease activity.
There was a high response rate from both phases of our surveys. Consequently, we feel that our survey respondents were representative of paediatric rheumatologists internationally.
One of the potential disadvantages of the Delphi technique is the lack of a uniform standardized methodology. There are several differing forms of Delphi described in the literature, including the modified Delphi, the policy Delphi and the real-time Delphi [1618]. Due to the flexibility of the technique, a frequent criticism is lack of methodological rigour. However, we feel that this flexibility was an intrinsic advantage as it enabled us to elicit a wide variety of responses from individuals without any peer interference. Regardless of the type of Delphi format used, the appropriate use of this technique requires a high degree of methodological precision, which we adhered to.
The other major criticism of the Delphi approach is that, since achieving consensus is a primary aim, there is a danger that possibly important variations in views will be concealed. Some experts have argued that considering that there is a strong natural tendency in the Delphi for opinion to centralize, resistance in the form of un-consensual distribution should be viewed with special interest [17]. It is possible that the items that we have selected are not, in some way, the best disease activity indicators; however, given the strong consensus, they represent the most widely held beliefs in the paediatric rheumatology community.
As the next step in the development of a disease activity tool, we plan to validate the current list of 29 items in a series of SJIA patients and determine the psychometric properties of each item. We have operationalized these items by formulating a preliminary working definition and scaling method for each; these definitions will be tested in the SJIA patient series. A final consensus phase will determine the best and most parsimonious list of items that can be used to determine disease activity in SJIA.
|
![]() |
Appendix 1 |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
![]() |
Acknowledgments |
---|
The authors have declared no conflicts of interest.
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|