Department of Rheumatology and 1 Pharmacy, Selly Oak Hospital, University Hospital Birmingham Foundation Trust, 2 Kingsfield Medical Centre, 3 Harborne Medical Practice, Birmingham, 4 Cannock Chase Hospital NHS Trust, Cannock and 5 St Mary's Hospital, Newport, Isle of Wight, UK.
Correspondence to: P. Jobanputra, Department of Rheumatology, University Hospital Birmingham Foundation Trust, Selly Oak Hospital, Raddlebarn Road, Selly Oak, Birmingham B29 6JD, UK. E-mail: P.Jobanputra{at}bham.ac.uk
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods. Patients newly referred to a university hospital were interviewed by a nurse using an agreed template. A multidisciplinary group of rheumatologists, nurse specialists, primary care physicians and a pharmacist reviewed current guidance and systematic reviews on NSAID use, and a series of interview transcripts. The group agreed, by informal consensus, important determinants of effective and safe NSAID use. Technical aspects of medical care and items that reflected interpersonal care were included in an index for assessing quality of care for individual patients. Interview transcripts of 100 patients were scored by panel members and reliability of scores was tested by calculating weighted percentage agreement and the statistic.
Results. Our final index had five domains: medical risk factors; steps taken to reduce risk; knowledge of adverse effects; NSAID dose; and cost efficiency. Each item was scored 0, 1 or 2. Scores were summed, giving a maximum of 10 (low scores indicating low quality). Intra-rater agreement was >90%; was 0.470.87 for individual domains and 0.59 for overall score. Inter-rater agreement for overall score was 95%;
was 0.250.78 for domains and 0.48 for overall score. Patients with especially low scores were identified using the mode of scores for five assessors; obvious clinical concerns were identified, supporting index face validity.
Conclusions. A simple index to evaluate quality of care for NSAID users based on a patient interview is described. This may be used by one or more assessors to examine care standards and highlight deficiencies in relation to NSAID use in practice.
KEY WORDS: Quality of care, NSAIDs, Assessment, Rheumatic diseases
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In Britain the quality of medical care is usually assessed by auditing aspects of clinical care against a standard (or guideline). Many guidelines and quality of care measures, for example the US Arthritis Foundation's quality indicator set [4, 5], are well crafted and allow for case-mix, but have a narrow focus and tend to be concerned with technical aspects of care, such as timely delivery of drug therapy, or use of specific interventions. Guidelines and quality measures are generally concerned with the processes of care, rather than outcomes. It is generally agreed that a focus on process, for many reasons, is appropriate [1, 6]. Disadvantages of many quality measures and guidelines (other than persuasive professional scepticism [79]) include difficulties in accommodating important aspects of care that are particularly relevant in chronic disease: for example, eliciting patients' views about medication [10]; allowing for variations in care because of comorbidity [11]; and other nuances of individual medical or social history, specifically the processes of caring [12, 13].
Despite wide appreciation of the benefits and potential toxicity of NSAIDs, the use of NSAIDs by many patients is suboptimal [14, 15]. In order to investigate this and to overcome limitations of guidelines on the use of NSAIDs, we set out to assess quality of care for patients with musculoskeletal pain. Our goal was to develop a tool that allowed clinicians to judge quality of care for NSAID users and that incorporated technical aspects of medical care and items that reflected interpersonal care.
![]() |
Methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Defining data on which to base quality assessments
A multidisciplinary study group of three rheumatologists, two clinical nurse specialists, two primary care physicians and a pharmacist reviewed preliminary interview transcripts and helped develop a semistructured questionnaire for interviewing patients. The group chose items that were clinically sensible and that reflected technical aspects of care, such as previous history of peptic ulceration, and items that could reflect interpersonal care. For example, the nurse enquired whether the patient knew any adverse effects of NSAIDs and also asked if the patient was happy with their NSAID; patient responses were transcribed and quoted verbatim, if appropriate. Clinicians making quality of care judgements used this text to assess care in the relevant domain; for example, stomach problems, described by a patient as an example of an adverse event, may be regarded as demostrating limited knowledge by the assessor.
The final interview template (shown in Appendix 1, available at Rheumatology Online) sought details of medical history, such as: symptoms of dyspepsia; reasons for NSAID use; comorbidity; concomitant medications, including over-the-counter preparations; knowledge and personal experience of the benefits and hazards of NSAIDs; and functional status. The nurse also sought patients' views about how well they were coping with pain, whether NSAID therapy had been discussed with their general practitioner, what they felt about tablet-taking and whether they would be prepared to increase the dose of their NSAID if appropriate. In a small number of cases a second nurse listened to telephone interviews, with consent, in order to check the accuracy of history-taking.
Agreeing items for inclusion in quality index
The study group relied on existing guidance and systematic reviews as a starting point [1619]; the group did not feel that an independent and exhaustive review of a well reviewed area of practice was needed. We agreed on important determinants of effective and safe NSAID use over seven 2 h meetings. It was agreed that goodness of NSAID use should be based on clinical judgements and that these judgements should be made, for individual patients, in the light of an abstract of an interview obtained by the nurse interviewer. Methods for evaluating optimal NSAID use were debated extensively and a final scoring system was devised, after several incarnations, through an iterative process that included independent evaluations of a series of interview transcripts.
Testing reproducibility of our index
In order to study the reproducibility or consistency of quality judgements, 100 patients (73 women and 27 men) were interviewed. Patients were drawn from 423 consecutive patients waiting for out-patient appointments of whom 98 (23%) responded to a mailing about the study and 194 patients who were approached in clinic. Most of the excluded patients were not taking an NSAID (180 patients) and others did not meet inclusion criteria, had already seen a specialist or declined to participate.
Data analysis
Six members of the study groupthree consultant rheumatologists, two general practitioners and one pharmacistscored each of 100 case histories independently. These 100 cases were assessed for inter-rater reliability and 30 cases, which were resubmitted randomly several weeks after the first evaluation, were rated again to study intra-rater consistency; five of the six assessors examined 30 cases twice. Case histories were submitted in batches of 10 and a minimum of 2 weeks elapsed between each batch. The original data were double-entered for accuracy. Consistency of scores is reported as the median weighted percentage of agreement and the median statistic, for observer pairs. Weighting of disagreements, to allow for partial agreement (as three or more levels of agreement were possible in each domain of our final index) was done using the squared deviation method [20]. StatsDirect, statistical software, was used for calculations.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
The group felt that knowledge of adverse effects provided an indication of inter-personal care and was important for the safe use of NSAIDs. Cost efficiency was also believed to be important as it could demonstrate care in selection of drug therapy; for example, use of a proton-pump inhibitor and Arthrotec might be regarded as cost-inefficient and to constitute thoughtless drug use.
Patient characteristics
The characteristics of 100 patients whose interview transcripts were used to test the reproducibility of our index are shown in Table 1. Single NSAIDs used were: ibuprofen, 28%; diclofenac alone, 20%; celecoxib, 13%; rofecoxib, 10%; Arthrotec, 8%; and a variety of other NSAIDs, 14%. Two NSAIDs were used by 3%, and low-dose aspirin and an NSAID by 4%. Nearly half (47%) of all patients had taken an NSAID for over a year. Over a third (35%) could not name any adverse events of NSAIDs but 53% cited potential gastrointestinal toxicity; 7% indicated that they had had no communications about the adverse effect of NSAIDs. Those who recalled information about NSAIDs cited package inserts (87%), their general practitioner (42%), the media (28%) and the internet (4%) as key sources of information.
Illustration of use of the index
We calculated the mode for each of the five domains shown in Table 2 for the 100 patients studied and whose interview transcripts had been assessed by at least five assessors. Values were then summed to give an overall quality score out of 10. This score was regarded as the gold standard assessment of quality as it relied on several expert opinions. The range of scores for these 100 patients is illustrated in Fig. 1. Four patients' (4%) cases scored less than 5 out of 10 and of these, one, a woman of 27 yr, was using two NSAIDs and had no awareness of potential adverse events. Two patients had a history of peptic ulcer or upper gastrointestinal bleeding and neither was using a proton-pump inhibitor or other appropriate gastrointestinal protective therapy, or a COX-2-selective NSAID. The fourth patient was using 225 mg of diclofenac daily and had alarming gastrointestinal symptoms, including nightly dyspepsia.
|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The judgements clinicians made were primarily process of care measures, such as steps taken to reduce the risk of NSAIDs and cost efficiency. But we also included items such as patient knowledge about risks and benefits of NSAIDs, which are important outcomes. We included items that we regarded as most pertinent to quality of care for NSAID users, whilst recognizing that items included in any index for clinical assessment were, inevitably, a compromise between comprehensiveness and usability. One of the domains included, medical risk factors, reflects patient characteristics that cannot be modified by providers, though perceptions of medical risk differed (Table 3). Therefore, low scores in this domain may unfairly reduce quality of care scores. This is an important limitation of our index, but, as NSAIDs are not completely irreplaceable medicines, especially in a heterogeneous population with musculoskeletal pain, use of NSAIDs in some patients with relevant medical risk factors may be less appropriate and thus their use could reflect poorer quality of care.
The key attributes of any new index are whether the index is consistent, accurate and fit for its purpose. These attributes are often described as various forms of validity [11]. Consistency of our index was appraised by studying inter- and intra-rater reliability. The reliability of quality of care judgements was similar to evaulations of diagnostic tests such as bone scans [22] and mammograms [23]. On discussing differences, assessors readily resolved disagreements in most cases. In some cases differences were due to factual errors; for example, not reading a case history with sufficient care or not appreciating drug cost differences. The scores shown in Table 3 indicate that using several independent evaluations, or a consensus method, is likely to be more reliable for quality of care evaluations. Accuracy of our index (criterion validity) cannot be ascertained since there is no reference standard or comparable quality of care index. Similarly, formal statistical comparisons without a comparator were unfeasible and construct validity (suitability of an index for its purpose) can only be examined by further field tests and inferred from inspection of included items, which were comparable to those of the Arthritis Foundation's quality indicator set for analgesic use [5]: informing patients about risks, gastrointestinal prophylaxis, selection of NSAID, and drug monitoring.
Our index was modelled on the Apgar score [21], which remains in use after many years, and like it relies on clinical judgements which we regarded as a strength of our instrument [11]. The use of an explicit list of criteria was considered but abandoned as it became increasingly burdensome as each new patient interview, requiring yet more items, was debated. The scope of explicit quality measures, proposed by the Arthritis Foundation [5], is wider than previous guidelines on NSAID use and includes recommendations on information provision and monitoring for toxicity. Included items, however, remain focused on technical aspects of care based on if-then-because statements. For example, if a patient is on NSAIDs daily and has risk factors for developing renal insufficiency, then serum creatinine should be measured at baseline and at least once in the first year of treatment, because NSAIDs are known to cause renal impairment. Our index demanded implicit judgements from assessors and allowed the consideration of wider clinical issues; for example, quality of care could be judged where an NSAID was being taken by a patient with established renal insufficiency.
One of the strengths of our methods was the use of a multidisciplinary group and reliance on data from patient interviews rather than hypothetical situations. Health professionals were chosen from a group of colleagues with a mutual interest but did not include gastroenterologists, renal physicians or patient representatives, omissions that may be regarded as important. A disadvantage of our method was a need to interview patients to prepare a transcript for quality of care evaluations. This limits the wide applicability of our index so that large-scale evaluations of quality of care would demand significant resources: though it is possible that the index could be adapted for use with a review of medical records or a patient questionnaire. Interviews rarely took longer than 15 min, except where patients had emotional needs or communication difficulties. Inaccurate recall of medical details is an obvious limitation of interviews, but this was compensated for by gaining an insight into patient knowledge, experienced benefits and adverse effects and attitudes to the use of NSAIDsall key parameters [24]. Interview records were not supplemented by a medical record review, though this may have improved data accuracy. However, it is clear that reliance on medical records alone underestimates quality of care [25].
Another important limitation was that we did not use formal methods of consensus to agree the items included in our index. Biases may have arisen from our informal methods [26]. For example, the differing status of health professionals, or individual personality traits, could have influenced the selection of items included or the format of our scoring method. There are, however, no ideal methods for achieving consensus and a spirit of mutual respect, many meetings, multiple opportunities for all participants to justify evaluations of clinical cases, and provision of expenses went some way to mitigating the limitations of an informal method of consensus.
In summary, our study focuses on people who use NSAIDs for musculoskeletal pain, not only those with a specific diagnosis, such as osteoarthritis. This overcomes problems of disease definition that are often ignored in guidelines and quality of care measures. We also include items that reflect elements of interpersonal care, not just those concerned with technical aspects of medical care, such as appropriate drug selection and monitoring. There are few tools to evaluate quality of care that include aspects of interpersonal care, and we believe the simple format and sufficient reliability of our index are an important step in quality of care evaluations in routine practice.
|
![]() |
Acknowledgments |
---|
This study was funded by an educational grant from Pfizer UK.
Supplementary data
Supplementary data are available at Rheumatology Online.
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|