a Department of Clinical Microbiology, Aarhus University Hospital, DK-8000 Aarhus C; b Department of Clinical Microbiology, Centralsygehuset i Slagelse, DK-4200, Slagelse, Denmark
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The aim of this study was to introduce a procedure for the evaluation and documentation of susceptibility test methods. The ability of four susceptibility test methods to detect decreased susceptibility to ciprofloxacin among 94 clinical isolates of Klebsiella pneumoniae was used as the test parameter. First, traditional regression analysis and frequency histograms were evaluated. Secondly, the accuracy of the methods was determined using the receiver operating characteristic (ROC) procedure. Finally, a calculation of the SN, SP, PV+ and PV for a range of inhibition zone diameters and MIC values was used to evaluate the robustness of the methods and to consider the consequences of an adjustment of inhibition zone or Etest MIC breakpoints.
![]() |
Materials and methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Ninety-four clinical isolates of K. pneumoniae with varying susceptibility to ciprofloxacin were selected at the Department of Clinical Microbiology, Aarhus University Hospital, Aarhus, Denmark.11 This material was not a random collection of clinical isolates but a selection with a preponderance of problem strains (decreased susceptibility to ciprofloxacin). The frequency of isolates with MIC 0.25 mg/L was 39%. Escherichia coli strain ATCC 25922 was used as a control for susceptibility testing including MIC determinations. As this strain has a very low MIC of ciprofloxacin (0.004 mg/L on MuellerHinton agar),12 K. pneumoniae isolates expressing stable higher MICs of ciprofloxacin were also used as controls.11
Antibiotics and discs
Bayer (Bayer A/S, Lyngby, Denmark) supplied ciprofloxacin as a standard powder. Paper discs for the prediffusion method (6 mm) were prepared in-house and contained 2.5 mg of ciprofloxacin or 30 mg of nalidixic acid. Discs from Oxoid Ltd (Sollentuna, Sweden) (6 mm) contained 5 mg of ciprofloxacin. Tablets (A/S Rosco, Tåstrup, Denmark) (10 mm) release 10 mg of ciprofloxacin as the diffusible amount. The Etest strip (AB Biodisk, Solna, Sweden) with ciprofloxacin consists of a continuum of two series of doubling dilutions, one ranging from 0.031 to 32 mg/L and one ranging from 0.023 to 24 mg/L. This ciprofloxacin gradient is applied to a plastic strip and the range of MIC values is dispersed on a scale measuring 45 mm.
Media
A basal nutrient beef broth medium,13 MacConkey agar plates,13 Danish blood agar plates13 and an IsoSensitest medium containing 41.4 g/L IsoSensitest powder (Oxoid Ltd) were from Statens Seruminstitut (Copenhagen, Denmark). MuellerHinton agar plates were made from distilled water and MuellerHinton powder (38 g/L) Difco (Bie & Berntsen A/S, Aarhus, Denmark).
Species identification
The isolates were identified in the laboratory using conventional methods with local media11 and re-identified using the API 20 E Bio typing system (bioMérieux, Marcy l'Étoile, France).
Reference method for MIC determination
The reference method for determination of MICs was an in-house agar dilution assay. Two-fold dilution series of antibiotic were prepared from a standard powder in distilled water. The dilutions were mixed into Mueller Hinton agar resulting in plates with final concentrations of ciprofloxacin ranging from 0.008 to 32 mg/L. Bacteria were cultured for 4 h in a nutrient broth. The culture was diluted and droplets containing c. 200 cfu were placed at the surface of the MuellerHinton agar plates (14 cm) using a multi-point pipette. This in-house method was used for technical reasons despite a lower inoculum as compared with NCCLS recommendations because an inoculum-dependent ciprofloxacin resistance mechanism (i.e. enzymic degradation or heterogeneous population of resistant mutants) among any of the selected K. pneumoniae strains was not observed in three earlier studies.
Susceptibility tests
Four susceptibility test methods were compared. The Oxoid disc diffusion method and the Rosco tablet diffusion method were carried out as described by the manufacturers (Oxoid Ltd; A/S Rosco). For the pre-diffusion method,13 antibiotic-containing discs were placed at the surface of an agar plate and kept at 5°C for 18 h (pre-diffusion period) before inoculation with bacteria. During the pre-diffusion period, an antibiotic concentration gradient is formed in the agar. Because this occurs before the initiation of bacterial growth, the influence of growth rate and inoculum on the zone size is diminished. The resulting inhibition zone diameters are relatively large compared with those obtained by the two other diffusion methods.13 The Etest was used following the instructions of the manufacturer (AB Biodisk). It is based on a concept similar to the pre-diffusion method. A plastic strip with a preformed concentration gradient of antibiotic is applied to the surface of an agar plate inoculated with bacteria. After incubation, the Etest MICs are read directly from the strip. For regression analysis, the measured Etest MICs and MICs determined with the agar dilution method were expressed on a logarithmic scale using the formula: log MIC = 9 + log2 (measured MIC value). Using this formula, the more detailed graduation of Etest MIC values is taken into account, making the calculation of the SN and SP and the robustness of the method more reasonable, especially when small alterations in susceptibility may be of importance.
Breakpoints for decreased susceptibility to quinolones
The chosen MIC breakpoint of decreased susceptibility to ciprofloxacin was 0.25 mg/L for the agar dilution method and the Etest. The ciprofloxacin inhibition zone breakpoints recommended by the manufacturers are: for the Rosco method 24 mm; the Oxoid method, 26 mm; and the pre-diffusion method, 35 mm. The inhibition zone breakpoint of decreased susceptibility to nalidixic acid is 17 mm for the pre-diffusion method.
Statistics and other calculations
Frequency histograms of MICs and zone diameters were constructed and a regression analysis of MICs on zone diameters was carried out using a computer program (StatGraphics, version 2.6, Rockville, MD, USA). Strains were considered incorrectly classified when determinations of MICs with the agar dilution method categorized the strains as decreased susceptible and determinations of zone diameters or Etest MICs categorized the same strains as susceptible, or vice versa. The correlation coefficients for the regression of zone diameters or Etest MICs on MIC values (agar dilution method) were calculated and compared for each method.
The ROC procedure was used to construct a ROC curve.14 Based on the selected MIC breakpoint for decreased susceptibility, the SN and SP of each of the susceptibility test methods were calculated successively, using each measured value (inhibition zone diameter in mm or Etest MIC in mg/L) as the cut-off value. The resulting SN values were plotted against the corresponding values of (1 SP), thus generating a ROC curve.
The SN represents the conditional probability that a K. pneumoniae with decreased susceptibility will be classified correctly by the test (true positive ratio). The SP represents the conditional probability that a susceptible K. pneumoniae will be classified correctly by the test (true negative ratio). The ROC curve thus shows the relationship between the true positive ratio (SN) and the false positive ratio (1 SP). Hence, the area under the curve (AUC) corresponds to the accuracy of a method.14 The ROC curve is independent of the prevalence of the test parameter,15 in this case the prevalence of decreased susceptibility. Furthermore, the SN and SP are plotted for a range of cut-off values. Hence, the SN (true positive ratio) increases with increasing cut-off value (breakpoint), while the SP (true negative ratio) decreases with decreasing cut-off value (breakpoint).
The PV+ and the PV depend on the prevalence (P) of the feature of interest, e.g. decreased susceptibility.15 In the present context, the PV+ represents the reliability of the finding of decreased susceptibility, while the PV represents the reliability of the finding of full susceptibility.15 The predictive values are calculated as follows:16
![]() |
and
![]() |
At given prevalences of decreased susceptibility of 5% and 15%, the PV+ and PV were calculated for each cut-off value (measured value of inhibition zone diameter or Etest MIC) and plotted together with the SN and the SP of the respective cut-off values. Based on this graph, the zone breakpoints for decreased susceptibility can be altered, improving either the SN, the SP or both, according to the purpose of the test.
The computer program Microsoft Excel 2000/Windows Millennium was used to design a spreadsheet with a table of the inhibition zones and MICs determined with the various methods and the formulae for calculation of SN, SP, PV+ and PV.
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
|
|
|
|
Susceptibility testing to nalidixic acid
Thirty-seven of the 94 K. pneumoniae isolates had ciprofloxacin MICs 0.25 mg/L determined with the agar dilution method. All of these 37 isolates and a further 12 ciprofloxacin-susceptible isolates were determined to have decreased susceptibility to nalidixic acid.
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
When introducing a new susceptibility test method in to a routine laboratory it is important to know the advantages and disadvantages of different methods in order to choose a strategy for training of technicians and for optimization of susceptibility testing in general. The four susceptibility test methods were chosen because they are very different and suitable for demonstration of the features of the evaluation procedure. The results were produced by technicians in a routine laboratory in order to expose the advantages and disadvantages of the methods illustrated in a daily routine setting.
Frequency histograms (Figure 1) showed that the use of the pre-diffusion, Etest and Rosco methods resulted in a more detailed distinction between the susceptible and decreased susceptibility isolates as compared with the Oxoid and agar dilution methods. The decreased susceptibility isolates appear as a separate population. However, in the Rosco method the breakpoint divided the population of decreased susceptible isolates (Figure 1
). In case of a high frequency of decreased susceptibility to ciprofloxacin among clinical isolates, a more detailed graduation or a confirmatory test may be warranted.
According to the regression analysis, all the methods had a relatively low correlation coefficient and several misclassifications of strains (errors). The main reason is the chosen MIC breakpoint (0.25 mg/L) and the susceptibility of the K. pneumoniae selected. Approximately one-third of the isolates have MIC values near the MIC breakpoint (Figure 1e
) and these isolates may randomly be classified as susceptible or decreased susceptible depending on minor variations in methodology. Consequently, errors may occur relatively frequently. Similar difficulties in detecting salmonellae isolates with ciprofloxacin MICs between 0.25 and 1 mg/L have been described.18 However, the relatively low correlation coefficients may also indicate the need for special training of laboratory staff, especially when introducing the Oxoid method.
An overall comparison of the ROC analysis (Figure 3) shows minor differences in the accuracy of the methods examined, but the robustness of the methods varies considerably according to the graphs showing the SN and SP (Figure 4
). The more horizontal the curves for the SN and SP at the recommended zone breakpoints, the more robust the method, as seen for the pre-diffusion and Etest methods, while for the Oxoid and Rosco methods relatively small variations in reading inhibition zone diameters would result in a more dramatic change, especially of the SN. Of note, the robustness of a method may vary with the antibiotics used and the bacterial species tested.
The Oxoid and Rosco methods misclassified several decreased susceptible isolates as fully susceptible (Figure 2a and b; Table 1
) giving highest priority to detection of susceptible K. pneumoniae, thus maximizing SP at the expense of SN at the recommended zone breakpoint (Figure 4a and b
; Table 2
). In contrast, the incorrect determinations of susceptible strains as decreased susceptible dominate for the pre-diffusion method and to a lesser extent for the Etest, maximizing detection of isolates with decreased susceptibility (Figure 2c and d
; Table 1
), e.g. maximizing SN at the expense of SP at the recommended breakpoint (Figure 4c and d
; Table 2
). Major errors affect the SN or SP slightly, but result in considerable decrease in PV+ depending on the breakpoint for decreased susceptibility as illustrated for the Oxoid method.
Displaying the PV+ and PV in Figure 4 makes it possible to optimize the choice of zone breakpoints according to different clinical and epidemiological situations taking the prevalence of decreased susceptibility into account. When a test is used to exclude decreased susceptibility or for screening purposes, it must be sensitive.16 Thus, when treating patients with severe infections or a compromised immune system2,3 (especially when the patients are infected with K. pneumoniae),19 a high SN for detecting decreased susceptibility should be favoured. Hence, the reliability of finding full susceptibility (PV) should be quite high. This would be accomplished when applying a zone breakpoint of 30 mm to the Oxoid method (Table 2
). On the other hand, the choice of a breakpoint resulting in a high SP but limited SN is appropriate when confirming a suspected decreased susceptibility.16 In the case of urinary tract infections, an overuse of newer broad-spectrum antibiotics may be avoided by preferring a high SP at the expense of SN, e.g. lowering the number of incorrect determinations of decreased susceptibility and increasing the reliability of the finding of decreased susceptibility (i.e. the PV+) for the antibiotics traditionally used against such infections. For the Oxoid method, this would for example be accomplished by using a breakpoint at 26 mm as opposed to the 30 mm optimizing the SN of the test (Table 2
).
According to the graphs in Figure 4, the inhibition zone or Etest MIC breakpoints may be altered, improving either the SN, SP, PV+ or PV, depending on the purpose of the test and other factors, for example the prevalence of decreased susceptible strains, the antibiotic policy chosen (e.g. to keep an antibiotic in the formulary), the resistance mechanisms of the bacterial isolates tested and different clinical and epidemiological situations. When there is a drift of antibiotic susceptibility, a correction of the breakpoint may be necessary in order to improve the use of a method.11
Ideally the PV+ and PV should both be high, and at presumed low resistance frequencies. Disc or tablet diffusion assays cannot always meet such requirements and the test results should generally be used with caution.
A confirmatory test for the determination of decreased susceptibility to ciprofloxacin may be applied when primary results are doubtful (e.g. determinations close to the breakpoint for decreased susceptibility). Resistance to nalidixic acid seems to predict decreased susceptibility of K. pneumoniae to ciprofloxacin as also described for salmonellae,18 although the rate of false decreased susceptible determinations is quite high. Alternatively, the Etest may be used as a confirmatory test for the determination of decreased susceptibility when primary results are doubtful,20 but the SN, SP, the prevalence of decreased susceptibility, and the PV+ and PV for such a combination of tests or a confirmatory test should be taken into account.
In conclusion, in the present study of decreased susceptibility to ciprofloxacin among K. pneumoniae, the accuracy and the robustness of susceptibility test methods can be used to expose the need for information and instruction of laboratory staff. Calculation of the SN, SP, PV+ and PV for a range of inhibition zones or Etest MICs makes it possible to optimize and justify the choice of inhibition zone breakpoints or Etest MIC breakpoints according to the SN and SP of the method employed. Furthermore, the consequences of adjustments of these breakpoints on the PV+ and PV can be analysed and related to different clinical and epidemiological situations. We believe that our approach can be used as a model for the evaluation and documentation of susceptibility test methods in general.
![]() |
Acknowledgements |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
Notes |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2
.
Thomas, J. K., Forrest, A., Bhavnani, S. M., Hyatt, J. M., Cheng, A., Ballow, C. H. et al. (1998). Pharmacodynamic evaluation of factors associated with the development of bacterial resistance in acutely ill patients during therapy. Antimicrobial Agents and Chemotherapy 42, 5217.
3 . Lorian, V. & Burns, L. (1990). Predictive value of susceptibility tests for the outcome of antibacterial therapy. Journal of Antimicrobial Chemotherapy 25, 17581.[Abstract]
4 . Dornbusch, K., King, A., Legakis, N. & The European Study Group on Antibiotic Resistance (ESGAR). (1998). Incidence of antibiotic resistance in blood and urine isolates from hospitalized patients. Report from a European collaborative study. Scandinavian Journal of Infectious Diseases 30, 2818.[ISI][Medline]
5 . Kahlmeter, G., Olsson-Liljequist, B. & Ringertz, S. (1997). Antimicrobial susceptibility testing in Sweden. IV. Quality assurance. Scandinavian Journal of Infectious Diseases Supplementum 105, 2431.[Medline]
6 . Ringertz, S., Olsson-Liljequist, B., Kahlmeter, G. & Kronvall, G. (1997). Antimicrobial susceptibility testing in Sweden. II. Species related zone diameter breakpoints to avoid interpretive errors and guard against unrecognised evolution of resistance. Scandinavian Journal of Infectious Diseases Supplementum 105, 812.[Medline]
7 . Threlfall, E. J., Ward, L. R. & Rowe, B. (1999). Resistance to ciprofloxacin in non-typhoidal Salmonellas from humans in England and Walesthe current situation. Clinical Microbiology and Infection 5, 1304.[Medline]
8 . Mølbak, K., Baggesen, D. L., Aarestrup, F. M., Ebbesen, J. M., Engberg, J., Frydendahl, K. et al. (1999). An outbreak of multidrug-resistant, quinolone-resistant Salmonella enterica serotype typhimurium DT104. New England Journal of Medicine 4, 14205.
9
.
Weigel, L. M., Steward, C. D. & Tenover, F. C. (1998). gyrA mutations associated with fluoroquinolone resistance in eight species of Enterobacteriaceae. Antimicrobial Agents and Chemotherapy 42, 26617.
10 . Frimodt-Møller, N., Højbjerg, T., Hvass, E., Møller, S., Mortensen, I. & Thomsen, V. F. (1985). Antibacterial activity in vitro and regression studies for ceftazidime and ceftriaxone. Acta Pathologica Microbiologica et Immunologica Scandinavica, Section B 100, 54352.
11
.
Schumacher, H., Scheibel, J. & Møller, J. K. (2000). Cross-resistance patterns among clinical isolates of Klebsiella pneumoniae with decreased susceptibility to cefuroxime. Journal of Antimicrobial Chemotherapy 46, 21521.
12 . Fuchs, P. C., Barry, A. L. & Brown, S. D. (1997). Is Escherichia coli ATCC 25922 useful for monitoring broth micro dilution tests of fluoroquinolones? Journal of Antimicrobial Chemotherapy 39, 5489.
13 . Schumacher, H., Bengtsson, B., Bjerregaard-Andersen, H. & Jensen, T. G. (1998). Detection of extended-spectrum ß-lactamases. Acta Pathologica Microbiologica et Immunologica Scandinavica 106, 979986.
14 . Castel, O., Grollier, G., Agius, G., Toullat, G. & de Rautlin de la Roy, Y. (1990). Evaluation of two media for antibiotic susceptibility testing of anaerobic bacteria using the receiver operating characteristic procedure. European Journal of Clinical Microbiology and Infectious Diseases 9, 66771.[ISI][Medline]
15 . Nettleman, M. D. (1988). Receiver operator characteristic (ROC) curves. Infection Control and Hospital Epidemiology 9, 3747.[ISI][Medline]
16 . Griner, P. F., Mayewski, R. J., Mushlin, A. I. & Greenland, P. (1981). Selection and interpretation of diagnostic tests and procedures. Annals of Internal Medicine 94, 553600.[ISI]
17 . Thomsen, K. S. & Sanders, C. C. (1994). Dissociated resistance among fluoroquinolones. Antimicrobial Agents and Chemotherapy 38, 2095100.[Abstract]
18
.
Hakanen, A., Kotilainen, P., Jalava, J., Siitonen, A. & Huovinen, P. (1999). Detection of decreased fluoroquinolone susceptibility in Salmonellas and validation of nalidixic acid screening test. Journal of Clinical Microbiology 37, 35727.
19 . Pedersen, G., Schønheyder, H. C. & Sørensen, H. T. (1997). Antibiotic therapy and outcome of monomicrobial Gram-negative bacteraemia: a 3-year population-based study. Scandinavian Journal of Infectious Diseases 29, 6016.[ISI][Medline]
20
.
Murphy, O., Fairbairn, C., Stewart, D. & Freeman, R. (1997). E-test evaluation of Enterobacteriaceae with reduced susceptibility to ciprofloxacin. Journal of Antimicrobial Chemotherapy 39, 5512.
Received 1 February 2001; returned 27 March 2001; revised 16 July 2001; accepted 26 July 2001