Evidence of effective and efficient teaching and learning strategies in the education of rheumatologist ultrasonographers: evaluation from the 3rd BSR musculoskeletal ultrasonography course

A. K. Brown, R. J. Wakefield, Z. Karim, T. E. Roberts2, P. J. O'Connor1 and P. Emery

Academic Unit of Musculoskeletal Disease, Department of Rheumatology, University of Leeds and 1 Department of Radiology, Leeds General Infirmary and 2 Medical Education Unit, School of Medicine, University of Leeds, Leeds, UK

Correspondence to: A. K. Brown, Academic Unit of Musculoskeletal Disease, Department of Rheumatology, University of Leeds, Leeds General Infirmary, Great George Street, Leeds LS1 3EX, UK. E-mail: andrewkbrown{at}ukonline.co.uk

SIR, Musculoskeletal ultrasonography (MUS) is an increasingly popular clinical tool in the hands of rheumatologists, with an expanding evidence base to support its use [1, 2]. However, the operator-dependent nature and level of technical expertise necessary to perform a proficient MUS assessment means that appropriate training is required [3]. Currently there is a lack of published data to guide rheumatologists who wish to be trained in MUS and an absence of an educational structure within which learning can take place [4, 5]. As part of an ongoing project to develop such an educational framework, we have developed an evidence-based, expert-derived rheumatology ultrasound curriculum containing educational outcomes that reflect both competency standards and clinical utility [6, 7]. We piloted our curriculum and competency-based educational approach at the recent British Society for Rheumatology (BSR) course on MUS, and would like to report the results of our evaluation, with the aim of providing helpful information to inform future MUS training and practice.

Informed by our expanding educational database, we introduced a number of specific learning and teaching strategies (Table 1), which have not been formally employed on previous courses. Fundamental to our approach, and reflecting the evolving nature of educating rheumatologists in MUS, was a commitment to evaluation. A previous report from the first BSR course examined perceptions of training and development of MUS in rheumatology practice [8]. We aimed to assess the efficiency and effectiveness of our educational strategy in the attainment of our predetermined learning outcomes [9]. We used the evaluation model developed by Kirkpatrick [10] with a two-level strategy employing qualitative and quantitative techniques. Level 1 aimed to ascertain learner perceptions and levels of satisfaction and thereby the efficiency of the educational process. This was measured by an anonymous structured questionnaire requiring free-text or graded responses (1–5 Likert scale), administered at the end of the course. Level 2 assessed any change in knowledge or skills, to determine the effectiveness of our strategies. This was measured by an anonymous written test consisting of multiple-choice and short-answer questions, taken immediately before (pre-test) and after the course (post-test). Pre-course educational material was withheld until the initial examination had been conducted, to maximize test validity. Ethical approval was not required for this study.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Specific educational strategies

 
Forty-six delegates attended the course; 77% were rheumatologists (51% specialist registrars, 35% consultants, 14% non-consultant career grades) and 23% from other disciplines (physiotherapy, podiatry, general practice, orthopaedics). Fifty-two per cent had no prior MUS experience, 41% reported some exposure and 7% confirmed regular MUS practice. Ninety-six per cent (level 1) and 80% (level 2) of delegates submitted completed evaluation forms.

The results from our level 1 evaluation demonstrated an excellent overall satisfaction rating (median Likert score 5/5; range 3–5) and confidence scores (median 4.5/5; range 3–5). Indeed, median scores ranged from 4/5 to 5/5 for all aspects of the educational process. Consistent themes were identified from the qualitative data. Areas reported to work well included: the practical emphasis of teaching; amount of time dedicated to hands-on scanning; small group size containing delegates with matched abilities, facilitating discussion and interpersonal learning; rotation between different tutors; exposure to an appropriate mix of normal MUS anatomy and pathology; timing and duration of teaching sessions. Areas for potential improvement included provision of additional precourse preparatory information; standardization of teaching sessions between tutors to ensure uniformity of information and teaching style to mirror information provided in the study guide; inclusion of a specific orientation session to provide instruction in machine operation and basic MUS anatomy, rather than integrating this into the general teaching. Delegates were asked to speculate as to the effect of the course on their future practice. The majority was motivated to continue to learn and develop their MUS skills. A number reported a desire to integrate and perform MUS as part of their clinical service. Others reported that they would seek financial support to purchase a machine and approach other ultrasonographers for educational mentorship.

Level 2 evaluation results revealed a mean precourse test score of 41% (S.D. 11.6; range 18–66%) and post-test score 64% (S.D. 10.4; range 38–78%). The mean change in score was +22% (S.D. 12.8; range +4 to +48%) (P<0.001).

This multidimensional evaluation of our pilot educational programme has yielded promising results and provides a preliminary endorsement of our approach. Our learning and teaching strategies appear to be well received by the target audience and have resulted in a demonstrable increase in knowledge and skills. This indicates an efficient and effective programme and provides good evidence for a potential model for future training. We intend to make minor adjustments to our curriculum in response to these data and perform a longitudinal evaluation in order to establish any future change in practice amongst delegates, to assess the long-term impact of our educational interventions. We hope that our experience provides helpful information that will facilitate future informed and constructive educational development in this expanding rheumatology discipline.

We would like to acknowledge the contribution of the delegates from the 3rd BSR musculoskeletal ultrasonography course. A.K.B. holds an Educational Research Fellowship awarded by the Arthritis Research Campaign.

The authors have declared no conflicts of interest.

References

  1. Grassi W, Cervini C. Ultrasonography in rheumatology: an evolving technique. Ann Rheum Dis 1998;57:268–71.[Free Full Text]
  2. Karim Z, Wakefield RJ, Conaghan PG et al. The impact of ultrasonography on diagnosis and management of patients with musculoskeletal conditions. Arthritis Rheum 2001;44:2932–3.[CrossRef][ISI][Medline]
  3. Wakefield RJ, Brown AK, O’Connor PJ, Karim Z, Grainger A, Emery P. Musculoskeletal ultrasonography: what is it and should training be compulsory for rheumatologists? Rheumatology 2004;43:821–2.[Free Full Text]
  4. Speed CA, Bearcroft PWP. Musculoskeletal sonography by rheumatologists: the challenges. Rheumatology 2002;41:241–2.[Free Full Text]
  5. Wakefield RJ, Brown AK, O’Connor PJ et al. Rheumatological ultrasound. Rheumatology 2003;42:1001.[Free Full Text]
  6. Brown AK, O’Connor PJ, Wakefield RJ, Roberts TE, Karim Z, Emery P. Practice, training and assessment amongst experts performing musculoskeletal ultrasonography: towards the development of an international consensus of educational standards for ultrasonography for rheumatologists. Arthritis Rheum 2004;51:1018–22.[CrossRef][Medline]
  7. Brown AK, O’Connor PJ, Roberts TE, Wakefield RJ, Karim Z, Emery P. Recommendations for musculoskeletal ultrasonography by rheumatologists: setting global standards for best practice by expert consensus. Arthritis Rheum 2005;53:83–92.[CrossRef][Medline]
  8. Speed CA, Bearcroft PWP. Training in musculoskeletal sonography: report from the first BSR course. Rheumatology 2002;41:346.[Free Full Text]
  9. Wilkes M, Bligh J. Evaluating educational interventions. Br Med J 1999;318:1269–72.[Free Full Text]
  10. Kirkpatrick DL. Evaluating training programmes: the four levels. San Francisco, CA: Berrett-Koehler, 1994.
Accepted 24 April 2005