External quality assessment of antimicrobial susceptibility testing in Europe

Jerry J. S. Snella,* and Derek F. J. Brownb

a Quality Assurance Laboratory, Central Public Health Laboratory, London NW9 5HT; b Public Health and Clinical Microbiology Laboratory, Addenbrooke's Hospital, Cambridge CB2 2QW, UK


    Abstract
 Top
 Abstract
 Introduction
 Materials and methods
 Results
 Discussion
 References
 
Comparability of results of antimicrobial susceptibility testing is essential for resistance surveillance studies. As different methods may be used in different countries, there may be particular problems with international comparisons of resistance rates. Data from external quality assessment (EQA) surveys participated in by laboratories from several European countries allow comparison of performance between countries. In this study, success rates with organism–antimicrobial agent combinations known to be difficult to test were examined. With penicillin resistance in pneumococci; vancomycin and high-level gentamicin resistance in enterococci; ampicillin, co-amoxiclav and chloramphenicol resistance in Haemophilus influenzae and methicillin resistance in staphylococci there were differences between countries in success rates for discrimination of resistant strains. This study suggests that differences between countries in rates of resistance for some organism–antimicrobial agent combinations should be interpreted with caution. International EQA is useful in the demonstration and clarification of such differences.


    Introduction
 Top
 Abstract
 Introduction
 Materials and methods
 Results
 Discussion
 References
 
Concern over increasing resistance to antimicrobial agents has highlighted the need for surveillance of antimicrobial resistance at a local, national and international level.13 There may be problems with comparison of results from different laboratories if different methods are used, and it is important to ensure that results are comparable. Comparison of results between countries requires particular caution as different testing methods may be used and different breakpoints for categories of susceptibility may be applied. One approach to defining differences between laboratories and moving towards harmonization of results is external quality assessment (EQA). EQA is the challenge of quality-assurance procedures by specimens of known but undisclosed content. Numerous national EQA schemes are available in Europe.4 The UK National External Quality Assessment Scheme (UK NEQAS) for microbiology was introduced in 1975 to serve the needs of UK clinical microbiology laboratories.5 The scheme covers a wide repertoire of clinical microbiology including antimicrobial susceptibility testing. Twelve strains a year are provided for this purpose and occasional extra specimens are provided in special surveys. In recent years, the scheme has been actively promoted throughout Europe resulting in increased participation. Participants within a country are provided with an analysis following each distribution of strains, showing overall performance, UK performance and performance of participants within their own country. These data provide an interesting insight into comparative performance between countries. Some strain–antimicrobial agent combinations are known to be particularly difficult to test,6 and data relating to some of these are presented here.


    Materials and methods
 Top
 Abstract
 Introduction
 Materials and methods
 Results
 Discussion
 References
 
Bacterial strains

Strains were provided by the Clinical Microbiology and Public Health Laboratory, Cambridge, UK (CMPHL) or the Antibiotic Resistance Monitoring and Reference Laboratory, Colindale, UK (ARMRL), and included isolates from recent infections, isolates referred for confirmatory testing or isolates with known resistance characteristics collected from other specialist laboratories. Freeze-dried strains were distributed to participants.7

Antimicrobial susceptibility testing

MICs were determined by an agar dilution method8 in two laboratories, either the Quality Assurance Laboratory, Colindale, UK (QAL) and CMPHL, QAL and ARMRL, or CMPHL and the Antimicrobial Chemotherapy Laboratory, City Hospital NHS Trust, Birmingham, UK. Disc susceptibility testing8 and, more recently, Etests (Cambridge Diagnostic Services, Cambridge, UK) were used in QAL to confirm the susceptibility of the strains before and after freeze drying. Strains were designated as susceptible or resistant to antimicrobial agents according to the criteria of the British Society for Antimicrobial Chemotherapy (BSAC).8 The widely used three category classification was used for penicillin susceptibility of Streptococcus pneumoniae.9,10

Information supplied to participants

A report form sent with the strains reported here specified the identity of the organism and the site of isolation as ‘other than urine’. More specific details were provided with enterococci, which were said to be isolated from cases of endocarditis where combination therapy was being considered. Participants were requested to test the strains by their routine methods and designate them as susceptible or resistant (or high-level resistant in the case of enterococci tested against gentamicin) to named antibiotics. A full analysis of participants' results was provided as described previously.5

Participants

The countries participating in the scheme and the number of participants in each varied for each distribution. The current distribution of participants by country is shown in Table IGo. In order to reduce the risk of misleading results from small numbers of laboratories, results for individual countries are presented here only when the number of participants was 10 or more. Confidentiality of results is an important underlying principle of the scheme. The numbers of participants in each country are published in the scheme's annual report; to prevent deductive disclosure of identity results are presented here as percentages of those testing an organism achieving the designated correct result. Each country is represented by the same arbitrary letter in all the tables of results (Tables II–IXGoGoGoGoGoGoGoGo); the letters have not been used in previous articles reporting on UK NEQAS results.


View this table:
[in this window]
[in a new window]
 
Table I.  Number of participants in the NEQAS general bacteriology scheme at September, 1999
 

View this table:
[in this window]
[in a new window]
 
Table II.  Detection of resistance to penicillin in seven strains of S. pneumoniae
 

View this table:
[in this window]
[in a new window]
 
Table III.  Detection of the VanB vancomycin resistance phenotype in five strains of enterococci
 

View this table:
[in this window]
[in a new window]
 
Table IV.  Reporting of susceptibility of enterococci to gentamicin for eight strains which were not high-level resistant
 

View this table:
[in this window]
[in a new window]
 
Table V.  Detection of resistance to ampicillin in eight ß-lactamase-negative strains of H. influenzae
 

View this table:
[in this window]
[in a new window]
 
Table VI.  Detection of resistance to co-amoxiclav in eight ampicillin-resistant ß-lactamase- negative strains of H. influenzae
 

View this table:
[in this window]
[in a new window]
 
Table VII.  Detection of resistance to chloramphenicol in six strains of H. influenzae
 

View this table:
[in this window]
[in a new window]
 
Table VIII.  Detection of resistance to methicillin in nine strains of S. aureus
 

View this table:
[in this window]
[in a new window]
 
Table IX.  Detection of resistance to methicillin in seven strains of coagulase-negative staphylococci
 

    Results
 Top
 Abstract
 Introduction
 Materials and methods
 Results
 Discussion
 References
 
Penicillin resistance in S. pneumoniae

The seven distributed strains with reduced susceptibility to penicillin would all be classified as moderately resistant, with MICs between 0.1 and 1 mg/L. Participants' reports of resistant, moderately resistant or intermediate were regarded as correct, and of susceptible as incorrect. There were apparent differences in performance between countries, with some consistently above the mean and some consistently below (Table IIGo). In seven countries the rate of detection of resistance for strains with MICs of 0.5–1 mg/L was >=90%. There is no evidence of consistent improvement in performance over time.

Vancomycin resistance in enterococci

Results for five strains of enterococci with the VanB phenotype and expressing low-level vancomycin resistance are presented in Table IIIGo. Detection of such resistance appears problematical, with resistance in the most recently distributed strain being detected by a mean of 77% of participants. The percentage of correct results was consistently higher than the mean in some countries and lower in others. There were obvious improvements in detection rates over time in eight of the 10 countries.

High-level gentamicin resistance in enterococci

High-level resistance to gentamicin was reliably detected in all countries (data not shown). In contrast, eight strains that were not high-level resistant but had gentamicin MICs of 4–16 mg/L, which is typical of enterococci, were reported as high-level resistant by a mean of 15% of participants. There was little consistency in such reporting (Table IVGo), although results from some countries show a tendency to be above or below the mean success rates. There was no evidence of improvement over time.

Non-ß-lactamase-mediated resistance to ampicillin and co-amoxiclav in Haemophilus influenzae

Resistance to ampicillin in eight ß-lactamase-negative strains was not readily detected in most countries between 1985 and 1998 (Table VGo). Such strains are resistant to co-amoxiclav, but spurious reports of susceptibility were also common (Table VIGo). Ability to detect such resistance varied between countries, with some consistently above or below the mean. There is obvious improvement over time in most countries and in the most recent distribution detection rates were 90% or above in five countries.

Chloramphenicol resistance in H. influenzae

Ability to detect chloramphenicol resistance in eight strains varied between countries with some consistently above or below the average (Table VIIGo). There did not appear to be any improvement over time.

Methicillin resistance in staphylococci

The level of performance has been uniformly high with methicillin-resistant Staphylococcus aureus, in 1999 detection rates were >=99% in every country (Table VIIIGo). There was more variation in performance with seven methicillin-resistant coagulase-negative strains (Table IXGo), with some strains apparently being easier than others: the detection rates for three strains were 100% in seven countries. Some countries appeared to achieve consistently higher detection rates than others, and in some there was evidence of improved performance over time.


    Discussion
 Top
 Abstract
 Introduction
 Materials and methods
 Results
 Discussion
 References
 
Overall performance in EQA susceptibility testing has been good6 and caution must be exercised in the interpretation of the results presented here, because they relate to those organism–antimicrobial combinations that have proved more challenging in repeated EQA distributions. Apparent differences in the EQA performance of participants from different countries may be due to several factors. In some countries, the majority of clinical laboratories participate in the UK NEQAS for Microbiology and the EQA performance should be more representative than with other countries, where only a small proportion of laboratories participate. With some combinations there is clearly association between time in the scheme and improvement in performance, although performance in countries where most participants have joined recently might be expected to be worse than in those with longer participation, and this was often not the case. It is encouraging that performance of some tests has improved with time, and this highlights the educational value of EQA. It is apparent, however, that some strains are more difficult to detect resistance in than others and inclusion of such strains may mask underlying improvements in success rates with more typical strains.

Although there were some differences between countries, the problem tests investigated generally caused difficulties in most European countries. In a previous report on performance in the UK, detection of penicillin resistance in S. pneumoniae appeared more difficult in strains with MICs of 0.1–0.25 mg/L than in those with MICs of 1 mg/L.6 However, this was not evident in most countries in the current study and all the strains tested presented problems in some countries. High-level resistance to vancomycin in strains of enterococci with the VanA phenotype has been detected easily in EQA specimens, but detection of low-level resistance in strains with the VanB phenotype has been problematic.11 Low-level resistance to vancomycin also caused problems throughout Europe in this study and similar difficulties have been reported in other European studies12 and in the USA.13 High-level resistance to gentamicin (MICs > 1000 mg/L) in enterococci has been detected easily in EQA specimens, but strains that are not high-level resistant have been falsely reported as resistant.11 In the current study similar errors were seen in all countries, although the mean success rate was 85%. Ampicillin- and co-amoxiclav-resistant, ß-lactamase-negative strains of H. influenzae are not reliably detected, although there does appear to be some improvement in performance over time. The level of resistance in these strains is low and the clinical consequences of such resistance remain unclear. Detection of chloramphenicol resistance in EQA samples of H. influenzae has proved difficult,14,15 and in most European countries in the current study some laboratories failed to detect this resistance. Failure rates of c. 15% in detection of resistance to methicillin in strains of S. aureus were common in early EQA distributions in the UK,6 but performance was good in all European countries in the current study. Detection of resistance in coagulase-negative staphylococci is more difficult and there are differences between species.16,17 These difficulties in testing coagulase-negative staphylococci were reflected in the results with some strains in the current study, the most recent strain distributed was difficult in that it showed a low level of resistance.

Associations have been shown between methodology and success with these more challenging organism/antimicrobial agent combinations.6 Several different methods are used in Europe and there are differences in MIC breakpoints for several agents.18 It is likely that differences in methods used in different countries have contributed to the differences in success noted and this will be investigated in the future.

Whatever the explanations for the observed differences in success rates between countries, such differences may distort comparisons of resistance rates and prevent a clear understanding of the incidence of antimicrobial resistance. Comparability of susceptibility testing results in different countries is essential for international surveillance, and international EQA has an important role to play in the demonstration and clarification of such differences.


    Acknowledgments
 
The authors thanks Mrs Jenny Andrews, Antimicrobial Chemotherapy Laboratory, City Hospital NHS Trust, Birmingham and the staff of the Antibiotic Resistance Monitoring and Reference Laboratory, Central Public Health Laboratory, Colindale for performing MIC determinations on some of the strains distributed.


    Notes
 
* Corresponding author. Tel: +44-20-8905-9890; Fax: +44-20-8205-1488; E-mail: organiser{at}ukneqasmic.win-uk.net Back


    References
 Top
 Abstract
 Introduction
 Materials and methods
 Results
 Discussion
 References
 
1 .  House of Lords Select Committee on Science and Technology. (1998). Resistance to Antibiotics and Other Antimicrobial Agents. The Stationery Office, London.

2 .  Huovinen, P. & Cars, O. (1998). Control of antimicrobial resistance: time for action. British Medical Journal 33, 613–4.

3 .  Livermore, D., MacGowan, A. P. & Wale, M. C. J. (1998). Surveillance of antimicrobial resistance. British Medical Journal 33, 614–5.

4 .  Uldall, A. (1995). World list of EQA organizers in laboratory medicine, list of key persons. EQA News 6, 32–41.

5 .  Snell, J. J. S., de Mello, J. V. & Gardner, P. S. (1982). The United Kingdom national microbiological quality assessment scheme. Journal of Clinical Pathology 35, 82–93.[Abstract]

6 .  Snell, J. J. S. (1994). Problems in susceptibility testing—findings of UK NEQAS for microbiology. Journal of Antimicrobial Chemotherapy 33, 1–4.[ISI][Medline]

7 .  De Mello, J. V. & Snell, J. S. (1985). Preparation of simulated clinical material for bacteriological examination. Journal of Applied Bacteriology 59, 421–36.[ISI][Medline]

8 .  Working Party Report of the British Society for Antimicrobial Chemotherapy. (1991). A guide to sensitivity testing. Journal of Antimicrobial Chemotherapy 27, Suppl. D, 1–50.[ISI][Medline]

9 .  Thornsberry, C. & Swenson, J. M. (1980). Antimicrobial susceptibility tests for Streptococcus pneumoniae. Laboratory Medicine 11, 83–6.

10 . Ward, J. (1981). Antibiotic-resistant Streptococcus pneumoniae: clinical and epidemiologic aspects. Review of Infectious Diseases 3, 254–66.[ISI][Medline]

11 . Snell, J. J. S., Brown, D. F. J., Perry, S. F. & George, R. (1993). Antimicrobial susceptibility testing of enterococci: results of a survey conducted by the United Kingdom National External Quality Assessment Scheme for Microbiology. Journal of Antimicrobial Chemotherapy 32, 401–11.[Abstract]

12 . Brown, D. F. J. & Courvalin, P. (1998). Quality assessment of glycopeptide susceptibility tests: A European collaborative study. European Glycopeptide Resistance Group. International Journal of Antimicrobial Agents 9, 153–63.[ISI]

13 . Tenover, F. C., Tokars, J., Swenson, J., Paul, S., Spitalny, K. & Jarvis, W. (1993). Ability of clinical laboratories to detect antimicrobial agent-resistant enterococci. Journal of Clinical Microbiology 31, 1695–9.[Abstract]

14 . Snell, J. J. S., Brown, D. F. J. & Phua, T. J. (1986). Antimicrobial susceptibility testing of Haemophilus influenzae: trial organised as part of United Kingdom National External Quality Assessment Scheme for Microbiology. Journal of Clinical Pathology 39, 1006–12.[Abstract]

15 . Snell, J. J. S., Perry, S. F. & Brown, D. F. J. (1991). Detection of chloramphenicol resistance in Haemophilus influenzae. Journal of Clinical Pathology 44, 77.[Abstract]

16 . Hussain, Z., Stoakes, L., Massey, V., Diagre, D., Fitzgerald, V., El Sayeed, S. et al. (2000). Correlation of oxacillin MIC with mecA gene carriage in coagulase-negative staphylococci. Journal of Clinical Microbiology 38, 752–4.[Abstract/Free Full Text]

17 . Andrews, J. M., Boswell, F. J. & Wise, R. (2000). Establishing MIC breakpoints for coagulase-negative staphylococci to oxacillin. Journal of Antimicrobial Chemotherapy 45, 259–61.[Free Full Text]

18 . Brown, D. F. J. (1994). Developments in antimicrobial susceptibility testing. Reviews in Medical Microbiology 5, 65–75.

Received 24 August 2000; returned 15 December 2000; revised 2 February 2001; accepted 22 February 2001