1 National Institute of Public Health and the Environment, Bilthoven; 6 Groningen University Hospital, Groningen, The Netherlands; 2 Institut Pasteur, Paris; 5 Institute de Vieille Sanitaire, Saint-Maurice, France; 3 PHLS, UK-NEQAS, London, UK; 4 Università di Verona, Verona, Italy
Received 12 February 2002; returned 2 May 2002; revised 10 July 2002; accepted 5 September 2002
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Antibiotic susceptibility of clinical isolates of bacteria is usually tested as part of routine laboratory investigations to establish the most adequate therapy for an infection. Detection of resistance relies on specimen collection from the patient, isolation, identification and susceptibility testing of the bacterial pathogen. Only recently a reference method for the determination of minimum inhibitory concentrations (MIC) has been proposed by the European Committee for Antimicrobial Susceptibility Testing (EUCAST),1 but there is still no European agreement on breakpoint criteria for interpreting the results into clinical categories [susceptible (S), intermediate (I), or resistant (R)]. As a result, methods for most agents still differ between countries, and interpretation of test results may differ.
The goal of this exercise was to organize external quality assurance of antibiotic susceptibility testing for laboratories participating in EARSS and to assess the comparability of susceptibility test results, as collected according to the EARSS protocol2 across countries and guidelines. Furthermore, this exercise assessed the comparability of MIC-yielding methods versus non-MIC-yielding methods (e.g. agar diffusion tests), and provided an overview of the frequency of use of various guidelines among EARSS laboratories. Quality assessment is essential in order for EARSS to assess the validity of comparing S. pneumoniae and S. aureus susceptibility data from a large number of laboratories from numerous countries and pooling it into a European database.
![]() |
Materials and methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The S. pneumoniae strains UA1283 and UA347 were intermediately resistant to penicillin G, and S. pneumoniae strain UA1449 was fully penicillin resistant. S. aureus strain UA1432 was homogeneously resistant to methicillin, and strain UA1450 was a heterogeneously resistant MRSA strain. The phenotypic expression of methicillin resistance of the S. aureus strains was analysed by performing two independent population analyses on agar plates containing different concentrations of the antibiotic, as described by Tomasz et al.3 The S. haemolyticus strain UA1434 was resistant to teicoplanin. The United Kingdom National External Quality Assurance Scheme (UK NEQAS) reference laboratory at the Central Public Health Laboratory, Colindale, London, organized the logistics of this study and arranged the shipment of the strains. The strains were prepared as freeze-dried cultures and sent by air-freight to EARSS national co-ordinating centres in 23 countries, who distributed the strains to the 471 laboratories participating in EARSS. Laboratories were asked to identify the control strains and to test them for susceptibility to specified antimicrobials using their routine procedures (for invasive specimens). They were asked to report the clinical categorization (S, I or R) and the MIC, if performed, as well as the breakpoints and guideline(s) followed. Five weeks were allowed for return of the results to UK NEQAS. Immediately after the closing date for return of results, brief details of the intended results were posted to participant laboratories, sent by e-mail to participants with e-mail addresses, and made available on the UK NEQAS web site. Laboratories received their individual results and a summary of the aggregate results.4 Where 10 or more laboratories within a country participated, tables of coded results specific to the country were produced.
Analysis comprised three parts: bacterial identification, antimicrobial susceptibility test results and the use of guidelines. We assessed participants results as being concordant or discrepant with the designated interpretation where all three reference laboratories agreed on the interpretation (S or I/R), and where the range of MICs of the reference laboratories allowed unambiguous interpretation by different guidelines. For assessing concordance we used only two categories: susceptible (S) versus non-susceptible [i.e. intermediate plus resistant (I/R)]. Results were assessed for correct interpretation of susceptibility/non-susceptibility for oxacillin, penicillin G and erythromycin against S. pneumoniae; for oxacillin, methicillin, gentamicin, vancomycin, teicoplanin and erythromycin against the S. aureus strains; and for gentamicin, vancomycin, teicoplanin and erythromycin against the S. haemolyticus strain.
In this study we considered results concordant if the reported interpretation of the participating laboratory agreed with the designated interpretation of the reference laboratories. The term concordance rate denotes the proportion of susceptibility tests with a correct result. For each countryexcept for France, Hungary and Malta, with only one laboratory participatingwe calculated the average concordance of participating laboratories. We also calculated for every guideline the average of the concordance of laboratories following that guideline, using Microsoft Excel (Microsoft Corporation, Release 97 SR-2; Redmond, WA, USA). We used SAS software (SAS Institute Inc., Release 8.01; Cary, NC, USA) for the calculation of the confidence intervals (CI), weighting the results for the number of tests performed in each country and considering that observations within one country are not independent.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
Strains were identified at the genus and species level. The three S. pneumoniae strains were correctly identified by: 425/428 (99%), 421/425 (99%) and 413/419 (99%) of the participating laboratories. Twelve laboratories from different countries did not identify one of the three S. pneumoniae strains correctly at species level and one laboratory failed to identify the genus correctly. Four laboratories identified one of the strains as Streptococcus mitis, four as Streptococcus viridans, two as Streptococcus sanguis, one as Streptococcus oralis, one as Streptococcus sp., and one as Aerococcus sp.
The two S. aureus strains were correctly identified by: 422/427 (99%) and 422/423 (100%) of the laboratories. Three laboratories identified one of the strains as coagulase-negative staphylococci, two as S. haemolyticus and one as Staphylococcus intermedius.
The S. haemolyticus strain was identified by 364/424 (86%) of the laboratories as S. haemolyticus or coagulase-negative staphylococcus and by 46/424 (11%) of the laboratories as staphylococcus species other than S. aureus. Thirteen laboratories (3%) misidentified the strain as S. aureus and one laboratory misidentified it as Enterococcus faecalis.
Antimicrobial susceptibility testing
Of the 8685 tests that were reported and assessed in this exercise, 8322 (96%) were interpreted correctly by the participants. The average of the concordance of all antimicrobial test results across countries surpassed 90% in all control strains (Figure 1). This figure shows for every control strain the average and the range across countries of the concordance of all antimicrobial test results that were assessed. The lower end of the ranges in the S. pneumoniae strains varied between 90% and 72%. In the S. aureus strains the lower ends ranged between 96% and 82%. We found similar results after stratification for guidelines (Figure 2), with the lowest concordance rate of 67% for one guideline for strain UA347.
|
|
|
|
|
Oxacillin, gentamicin, erythromycin, teicoplanin and vancomycin susceptibility in S. aureus. The overall concordance for detection of oxacillin (i.e. methicillin) resistance in the homogeneously resistant S. aureus strain UA1432 was 100%. The overall concordance for the heterogeneously resistant MRSA strain UA1450 was much lower, at 77%. Three countries (Czech Republic, Greece and Iceland) yielded notably lower concordance rates (Table 6), but no difference was found among different guidelines followed in Europe (data not shown).
|
Respectively 100% and 99% of the participants detected erythromycin and gentamicin resistance in the homogeneously resistant MRSA strain. Erythromycin and gentamicin susceptibility in the heterogeneously resistant MRSA strain was interpreted correctly by 98% and 99% of the participants, respectively. For teicoplanin susceptibility in the heterogeneously resistant MRSA strain, we found a concordance rate of 100%.
Teicoplanin, vancomycin, gentamicin and erythromycin susceptibility in S. haemolyticus. For detection of teicoplanin resistance in the S. haemolyticus strain, the overall concordance rate was 82% (Table 7), with five countries (Belgium, Denmark, Luxembourg, The Netherlands and UK) scoring low concordance rates. Vancomycin susceptibility of this strain was interpreted correctly by 94% of the participants. Gentamicin and erythromycin resistance were detected by 97% and 99% of participants, respectively. The overall concordance for detection of oxacillin susceptibility in the S. haemolyticus strain was 83%, with three countries (Bulgaria, Israel and Slovenia) clearly scoring lower.
|
|
For the two S. aureus strains, oxacillin MIC methods (n = 363) reached 99% concordance in the homogeneously resistant MRSA, and 76% concordance in the heterogeneously resistant MRSA. Other methods (n = 405) yielded a concordance of 100% and 78%, respectively.
Use of guidelines
Of the 395 laboratories specifying which guideline they used, 242 (61%) in 19 countries followed the NCCLS guideline (Table 9). Any other guideline was not followed by more than 6% of the laboratories, in at most two countries. With only one reference laboratory participating in France, and Norway not participating, the CA-SFM and Norwegian Working Group on Antibiotics (NWGA) guidelines were not represented.5 Thirty-eight of the 433 laboratories (9%) did not specify the guideline they followed.
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In this exercise, 8685 tests were reported and assessed but some 850 more results were expected. Laboratories were asked to test all antimicrobial agents listed on the report form, but in case they normally test for another agent from the same class they were asked to specify the name of this agent in the same box. This may have given rise to misunderstanding. Laboratories were asked furthermore to test the susceptibility using routine procedures. Apparently this was interpreted by a number of laboratories to test only those organismantimicrobial combinations that they test routinely. Laboratories should be solicited in future QA exercises to test and report all the requested organismantimicrobial combinations.
To screen for penicillin resistance in S. pneumoniae, almost all participants used the oxacillin 1 µg disc, achieving a very high concordance rate. This indicates that the oxacillin screen disc reliably discriminates susceptible from non-susceptible strains.
The concordance for penicillin resistance in S. pneumoniae is somewhat lower when laboratories test for penicillin G. This lower concordance rate is partly due to the fact that a substantial number of laboratories use non-MIC-based penicillin confirmation techniques. Indeed, we observed a far higher concordance (94%) for quantitative methods yielding a penicillin MIC than for non-MIC-yielding methods (79%), confirming the rationale of the EARSS protocol to perform MIC determination on S. pneumoniae strains found to be non-susceptible by a screen test. The difficulty of laboratories using disc diffusion tests to recognize reduced penicillin susceptibility in S. pneumoniae has recently been described in another international QA survey.6
The differences in concordance among the three S. pneumoniae control strains are a reflection of how many dilution steps the penicillin MIC for the strain is distant from the susceptible breakpoint. Indeed, more laboratories misinterpreted strain UA347 as being penicillin susceptible than strain UA1449. Some guidelines yielded lower concordance rates for the determination of penicillin resistance, like the DIN guideline as well as the guidelines specified under Other in Table 5. For the DIN guideline, this may be related to the susceptible breakpoint, which is one dilution step higher. Almost all national guidelines in Europe, as well as the NCCLS guideline, consider isolates of S. pneumoniae to be non-susceptible to penicillin if the MIC is >0.06 mg/L.711 The DIN guideline considers isolates to be non-susceptible to penicillin if the MIC is >0.12 mg/L.12 However, it should be noted that the DIN, as well as the guidelines specified under Other, were used only by relatively small numbers of laboratories, allowing for larger variation. All three S. pneumoniae control strains were non-susceptible, and the high concordance rates represent a high sensitivity of EARSS laboratories to detect penicillin non-susceptibility in S. pneumoniae. It is not possible from this exercise to infer the specificity of EARSS laboratories to detect penicillin susceptibility in S. pneumoniae. Because all three S. pneumoniae strains were highly resistant to erythromycin, they were not really a challenge to participating laboratories. Virtually all laboratories correctly determined erythromycin resistance.
For the detection of oxacillin resistance in S. aureus, we included one strain that was homogeneously resistant and another strain that was heterogeneously resistant to oxacillin. Laboratories performed extremely well in detecting oxacillin resistance in the homogeneously MRSA strain, but the concordance rate dropped from 100% to 77% in the heterogeneously resistant MRSA strain. A notable proportion of laboratories in three countries failed to detect the heterogeneously resistant MRSA strain. However, although we observed differences in concordance among countries, we found no significant differences among guidelines. Detecting heterogeneously resistant MRSA possibly depends more on test methods used by individual laboratories than on differences in guidelines. Laboratories in most countries, and in some countries in particular, should scrutinize carefully their capability of detecting low-frequency resistant subpopulations, and ensure that proper laboratory methods are used to detect heterogeneously resistant MRSA strains.
Detection of glycopeptide resistance in staphylococci is of paramount importance. For safety reasons we chose not to distribute a vancomycin intermediate (or resistant) strain among laboratories all over Europe, but instead distributed a S. haemolyticus strain that was resistant to teicoplanin. Vancomycin susceptibility of the two S. aureus control strains was interpreted correctly by participating laboratories, but teicoplanin resistance of the S. haemolyticus strain was often missed. Quantitative methods yielding an MIC were more frequently concordant than non-MIC methods for the detection of teicoplanin resistance against the S. haemolyticus strain (91% versus 71%).
Only a few participants misinterpreted gentamicin and erythromycin susceptibility in staphylococci, indicating that most participating laboratories are capable of determining gentamicin and erythromycin resistance.
This exercise provides a good overview of the guidelines being followed in Europe, with exception of the French and Norwegian guidelines. The NCCLS guideline is widely followed in Europe. In 10 countries NCCLS seems to be the only guideline in use; but in the countries that have issued national guidelines (Germany, The Netherlands, Sweden and Spain) some laboratories also follow the NCCLS guideline. The BSAC and Swedish Reference Group for Antibiotics (SRGA) guidelines are the only European guidelines used in more than one country. Because France and Norway are not represented in this study, we cannot infer on the use of guidelines there. It is probable, however, that the CA-SFM and NWGA guidelines are not widely followed in other European countries.
We found that 9% of participating laboratories did not specify which guideline they follow. Apart from the obvious reason that some laboratories simply may not have reported which guideline is followed, it may also be that some laboratories use in-house guidelines, or other non-documented guidelines. An overview of the antimicrobial susceptibility test breakpoints of national societies has been published recently.13 The authors recommend that other national guidelines (e.g. Czech 98) are also documented in international literature and that every laboratory works according to well-documented guidelines so that susceptibility test results are reproducible and comparable. Guidelines should also be freely accessible through the Internet.
It should be noted that overall the breakpoints defining susceptibility or resistance of bacteria to antimicrobial agents do not differ greatly between guidelines used in Europe. Baquero14 argued that it is possible to establish a theoretical consensus standard list of breakpoints, such that more than 95% of the breakpoints proposed by the different systems differ from the consensus standard by no more than one dilution.
We hope that these findings may add to the process of standardizing breakpoints in Europe as brought forward by the EUCAST.
It is shown that, overall, countries participating in EARSS are capable of delivering susceptibility data of good quality. The comparability of susceptibility data for penicillin resistance in S. pneumoniae and for homogeneous methicillin resistance in S. aureus is satisfactory among European countries and across guidelines. However, we emphasize the importance of determining an MIC for suspected penicillin non-susceptible S. pneumoniae and for suspected glycopeptide non-susceptible S. aureus.
Laboratories, particularly in some countries, may need to improve their capability of detecting oxacillin resistance in heterogeneously resistant MRSA and teicoplanin resistance in S. haemolyticus.
A number of laboratories did not fill out the form completely, for example by not reporting the species identification or not performing all the susceptibility tests requested. Not doing (or reporting) a test is considered as non-performance and hinders the assessment of the performance of laboratories. This should be avoided in future QA studies by organizers and participants.
Not every laboratory produced good results in this exercise, and the performance of some individual laboratories could probably be improved. For continuous external quality assessment we recommend that laboratories participate in national and international schemes with frequent distributions of control strains.
However, we feel reassured by this exercise that overall the antimicrobial susceptibility testing data as monitored through the national surveillance systems that participate in EARSS are of good quality. It is hoped that laboratories participating in this EARSSUK NEQAS quality assurance are encouraged to maintain and improve their performance, as has been observed in other surveillance schemes.15,16
![]() |
Acknowledgements |
---|
Participating countries and national representatives in EARSS during 2000: Austria, H. Mittermayer, W. Koller; Belgium, H. Goossens, F. van Loock; Bulgaria, B. Markova; Czech Republic, P. Urbaskova; Denmark, T. L. Sørensen, D. Monnet; Finland, P. Huovinen, O. Lyytikäinen; France, P. Courvalin, H. Aubry-Damon; Germany, W. Witte, T. Breuer; Greece, N. Legakis, G. Vatopoulos; Hungary, M. Konkoly-Thege; Iceland, K. Kristinsson, H. Briem; Ireland, O. Murphy, D. OFlanagan; Israel, R. Raz; Italy, G. Cornaglia, M. L. Moro; Luxembourg, R. Hemmer; Malta, M. Borg; Netherlands, A. de Neeling, W. Goettsch; Norway, E. Hoiby, P. Aavitsland; Poland, V. Hryniewicz; Portugal, M. Caniça, M. Paixao; Slovenia, M. Gubina; Spain, F. Baquero, J. Campos; Sweden, B. Olsson-Liljequist, O. Cars; United Kingdom, A. Johnson, M. Wale.
![]() |
Footnotes |
---|
Present address. Robert Koch-Institute, Berlin, Germany.
¶ Participating countries and national representatives in EARSS during 2000 are listed in the Acknowledgements.
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2 . EARSS website. (2001). EARSS Manual. EARSS management Team. [Online.] http://www.earss.rivm.nl (21 June 2002, date last accessed).
3 . Tomasz, A., Nachman, S. & Leaf, H. (1991). Stable classes of phenotypic expression in methicillin-resistant clinical isolates of staphylococci. Antimicrobial Agents and Chemotherapy 35, 1249.[ISI][Medline]
4 . UK-NEQAS website. (2000). Report of External Quality Assessment Exercise EARSS-NEQAS 2000. UK-NEQAS. [Online.] http://www.pcug.co.uk/~ukneqasm/ and at http://www.earss.rivm.nl (21 June 2002, date last accessed).
5 . Bergan, T., Bruun, J., Digranes, A., Lingaas, E., Melby, K. & Sander, J. (1997). Susceptibility testing of bacteria and fungi. Report from the Norwegian working group on antibiotics. Scandinavian Journal of Infectious Diseases, Suppl. 1, 103.
6
.
Tenover, F. C., Mohammed, M. J., Stelling, J., OBrien, T. & Williams, R. (2001). Ability of laboratories to detect emerging antimicrobial resistance: proficiency testing and quality control results from the World Health Organizations external quality assurance system for antimicrobial susceptibility testing. Journal of Clinical Microbiology 39, 24150.
7 . The British Society for Antimicrobial Chemotherapy (BSAC) website. (1998). Standardized disc sensitivity testing method, The Newsletter of the British Society for Antimicrobial Chemotherapy, 1998. BSAC, Birmingham, UK. Update available online at: http://www.bsac.org.uk/discdiff/sensitivity2.pdf (21 June 2002, date last accessed).
8 . Commissie Richtlijnen Gevoeligheidsbepalingen. (1996). Nederlands Tijdschrift voor Medische Microbiologie 4, 5.
9 . Cars, O. (Ed.) (1997). Antimicrobial susceptibility testing in Sweden. Scandinavian Journal of Infectious Diseases, Suppl. 1, 105.
10 . Société Française de Microbiologie. (1996). Report of the Comité de lAntibiogramme de la Société Française de Microbiologie. Clinical Microbiology and Infection 2, Suppl. 1, S149.[Medline]
11 . MENSURA. (2000). Recomendaciones del grupo MENSURA para la selección de antimicrobianos en el estudio de la sensibilidad y criterios para la interpretación del antibiograma. Revista Española Quimoterapia 13, 7386.
12 . Deutsches Institut für Normung. (1998). Methods for the Determination of Susceptibility of Pathogens (Except Mycobacteria) to Antimicrobial Agents. MIC Breakpoints of Antibacterial Agents, Suppl. 1, pp. 589404. DIN, Berlin.
13 . Degener, J. E. & Phillips, I. (2001). Comparison of antimicrobial susceptibility test breakpoints of national societies. Clinical Microbiology and Infection 7, 514.[ISI][Medline]
14 . Baquero, F. (1990). European standards for antibiotic susceptibility testing: towards a theoretical consensus. European Journal of Clinical Microbiology and Infectious Diseases 7, 4925.
15 . Forster, D. H., Krause, G., Gastmeier, P., Ebner, W., Rath, A., Wischnewski, N. et al. (2000). Can quality circles improve hospital-acquired infection control? Journal of Hospital Infection 45, 30210.[ISI][Medline]
16
.
Snell, J. S. & Brown, D. F. J. (2001). External quality assessment of antimicrobial susceptibility testing in Europe. Journal of Antimicrobial Chemotherapy 47, 80110.