Laboratory testing policies and their effects on routine surveillance of community antimicrobial resistance

Margaret L. Heginbothom1, J. T. Magee1, Joanna L. Bell1, F. D. J. Dunstan2, A. J. Howard1, Sharon L. Hillier2, S. R. Palmer2 and B. W. Mason1,* on behalf of the Welsh Antibiotic Study Group§

1 National Public Health Service for Wales, Abton House, Wedal Road, Cardiff CF14 3QX; 2 Department of Epidemiology, Statistics and Public Health, University of Wales College of Medicine, Cardiff CF14 4XW, UK

Received 2 October 2003; returned 8 December 2003; revised 7 March 2004; accepted 9 March 2004


    Abstract
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Objective: To investigate the effects of laboratory testing policies, particularly selective testing, rule-based reporting and isolate identification, on estimates of community antimicrobial resistance.

Materials and methods: Antibiotic resistance estimates were analysed from an all-Wales dataset for approximately 300 000 community isolates of common pathogens.

Results: Selective testing policies were often associated with markedly increased resistance, particularly for second-line testing. Site-specific testing tended to yield variant resistance estimates for eye and ear isolates. Estimates from rule-based reporting deviated markedly from test-result-based reporting. Urinary isolates reported as Escherichia coli showed greater susceptibility than those reported as undifferentiated urinary ‘coliforms’. The proportion of isolates tested for an antibiotic by a laboratory was a useful indicator of selective testing in this dataset. Selective testing policies had invariably been applied where the proportion of isolates of a species tested against an antibiotic was <90%. As this proportion fell with increasingly selective policies, divergence from pooled-all-Wales non-selective estimates tended to increase, with a bias to increased resistance.

Conclusions: Selective testing, rule-based reporting and urinary coliform identification policies all had significant effects upon resistance estimates. Triage based upon the proportion of isolates tested seemed a useful tool in assigning analysis resources. Where <20% of isolates were tested, selective policies with inherent bias to increased resistance were common, the low number of isolates gave high potential sampling errors, and little confidence could be placed in the resistance estimate. Where 20–90% of isolates were tested, detailed analysis sometimes revealed resistance estimates that might be usefully retrieved. Where >=90% of isolates were tested, there was no evidence of selective testing, and inter-laboratory variation in estimates appeared to be safely ascribable to other effects, e.g. methodology or real variation in resistance levels.

Keywords: community isolates, resistance estimates


    Introduction
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
There is a lack of confidence in aggregate data from routine reporting of antibiotic resistance in local laboratories that discourages local surveillance, resulting from criticism that the data are subject to a range of potential biasing factors. These include: non-standardized antimicrobial susceptibility testing; inclusion of estimates based on selective testing; rule-based reporting; and failure to identify isolates to species level. Uncertainty about the validity of resistance estimates is one of several factors that discourage proactive participation in surveillance of resistance. This uncertainty also reduces confidence in interpretation of surveillance results, and so discourages intervention. These uncertainties would not be a feature of a larger surveillance programme based on standardized methodology.

The Welsh Antibiotic Study Group is involved in a major survey of resistance in community infections. Data for approximately 300 000 routine community isolates throughout Wales from 1996 to 2001 have been collected retrospectively, to generate hypotheses, and data for 2001 to 2003 will be collected to test these hypotheses. The aims of the survey include publication of definitive information on the effects of potential biasing factors on susceptibility estimates derived from surveillance of routine isolates. The retrospective data were particularly suited to analysis for the effects of differences in laboratory policy, as it covered a period pre-dating any major shift to inter-laboratory standardization.

Susceptibility estimates should ideally reflect the expected prevalence of resistant organisms, to allow good predictivity of success of blind therapy. Selective sampling by clinicians may bias estimates from routine diagnostic data but this might be amenable to correction from sentinel studies.1 Nonetheless, estimates from routine data reflect susceptibilities for a population that can be readily identified by practitioners—those patients where a decision to seek laboratory investigations has been taken—and so contain information useful in prediction. However, further effects may degrade their relevance. This paper documents analyses for three areas where laboratory-testing policies may affect community susceptibility estimates—selective testing, rule-based reporting, and species identification of urinary coliforms. We were particularly interested in the biasing effects of the various antibiotic testing/reporting policies defined below.

Selective testing policies comprised:

1 Site-specific testing, where an antibiotic is only tested against isolates of a species that originate from specific body sites. For example, some laboratories test tetracycline for Staphylococcus aureus from topically treated sites such as ear swabs, but not other sites.
2 Direct-susceptibility testing, where urine specimens are plated directly for susceptibilities, without prior isolation and identification. The antibiotics tested may not be those normally tested for the species (e.g. S. aureus susceptibilities reported for a urinary coliform antibiotic set).
3 Second-line testing, where a species is only tested with an antibiotic if the isolate shows resistance to other antibiotics in a routine initial test set (e.g. cefuroxime was tested selectively for multi-resistant urinary coliforms by most laboratories in this survey).
4 Prescribing-specific reporting, where susceptibility to the antibiotic is only reported for the species if the specimen request form states that the patient is being treated with the antibiotic. The result may be derived from a laboratory test or from a rule-base.
5 Matched-susceptibility reporting, where several pathogens have been isolated from a specimen, and results for antibiotics not normally tested for the species are reported as an indication of therapy for the mixed infection. For example, some laboratories report flucloxacillin susceptibility for Streptococcus pyogenes in a mixed infection with penicillin-resistant methicillin-susceptible S. aureus (MSSA), or co-amoxiclav for Streptococcus pneumoniae in mixed infection with a ß-lactamase-producing Haemophilus influenzae. Again, the result may be derived from a laboratory test or from a rule-base.

In addition, we were interested in the frequency of changes in testing policies, and the effects of these changes on susceptibility estimates.

Rule-based reporting involves results that are derived from a set of rules defining invariant ‘results’ for specific antibiotics and species, irrespective of any laboratory test. This may be applied to all isolates of a species (e.g. some laboratories reported all H. influenzae isolates as intermediate or resistant to erythromycin), or to specific isolates (e.g. S. pyogenes isolates might be reported as flucloxacillin-susceptible under specific circumstances—see 4 and 5 above).

Inter-laboratory variation also occurs in the extent of identification. For urinary coliform isolates, all lactose fermenting isolates may be reported as coliforms, or further identification may distinguish Escherichia coli isolates from other coliforms. It is widely believed that inclusion of coliforms other than E. coli biases susceptibility estimates to increased resistance.

Several of these policies involve selection of isolates that might show resistance, or are so severely selective that the small numbers yield unreliable resistance estimates. Resistance estimates derived from such selective testing may be a poor basis for prediction of outcome in blind therapy.


    Methods
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Isolate data

Routine susceptibility data for Welsh community isolates of urinary coliforms, MSSA and methicillin-resistant S. aureus (MRSA), H. influenzae, S. pneumoniae and S. pyogenes were analysed for the 5 years from April 1996 to March 2001. The data were gathered from 14 of the 18 Welsh and Welsh border laboratories serving Welsh practices (see Acknowledgements); four could not provide data because of various IT problems. The data comprised, for each isolate: date of specimen collection; specimen type and site; patient date of birth, identifier and gender; practice code; laboratory number; and susceptibility (where tested) to 13 antibiotics (penicillin, ampicillin/amoxicillin, co-amoxiclav, cefuroxime, cefalexin, flucloxacillin, erythromycin, tetracycline, trimethoprim, nalidixic acid, ciprofloxacin, norfloxacin and nitrofurantoin). These data were obtained from the proprietary laboratory computer system used in the 14 laboratories and imported into Microsoft Excel. Visual Basic macros were used to format the data. The data were then assembled in organism groups (urinary coliforms including those identified as E. coli, MSSA, MRSA, H. influenzae, S. pneumoniae and S. pyogenes) and transferred to Statistical Package for the Social Sciences (SPSS Inc., Chicago, IL, USA) for analysis. Duplicate isolates, defined as isolates occurring less than 92 days after a prior indistinguishable organism from the same patient, were eliminated from the data before analysis.

The methods used by laboratories to test susceptibilities were based on disc diffusion. All laboratories involved participated in the United Kingdom National External Quality Assessment Service scheme and attained acceptable results. Information on the methods used for the identification of urinary coliform isolates was obtained from a telephone survey of laboratories.

‘Selective susceptibilities’

For each species, the total number of isolates available for testing at each laboratory was noted. The number of isolates tested by the laboratory was determined for each antibiotic, and the percentage of isolates tested (PT) was recorded for that laboratory/species/antibiotic combination. No further analysis was undertaken if <30 isolates had been tested. For 30 isolates, if 15 were susceptible, the 95% confidence limits on the resistance estimate would be 33.2–66.9%; if none were resistant, the limits would be 0–11.4%. It was felt that levels of uncertainty beyond this would remove any confidence in the estimates. The 95% confidence intervals for proportions and the difference between proportions were calculated by the method of Wilson.24

For the analysable data, details of isolates that had been tested against the antibiotic were compared with those that had not. This comparison sought consistent patterns in: date interval for specimen submission (indicating a change in testing policy); specimen site differences (indicating site-specific testing); and multiple-resistance to other antibiotics (indicating second-line testing of resistant isolates). Laboratories were contacted to confirm the deductions, obtain information on rule-based reporting, and to resolve less obvious patterns of selective testing. The latter were generally due to either prescribing-specific testing, or matched susceptibility reporting.

Pooled non-selective all-Wales susceptibility estimates were calculated for comparison with individual laboratory estimates. For the purposes of this paper, these were taken from pooled data for those laboratories testing for >=90% of isolates for the organism/antibiotic, provided there was more than one such laboratory and that the results from these laboratories were test-based rather than rule-based. Analyses were also carried out for the population subdivisions recognized in the health services organization for Wales, five Health Authorities, which are further divided into a total of 22 Local Health Groups that are coterminous with unitary local authorities.


    Results
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Selective susceptibility testing policies

The 14 laboratories, six organisms and 13 antibiotics gave 1092 possible laboratory/organism/antibiotic combinations. No data were available for 16 combinations. For 457, the number of isolates tested was <30; these ‘minimal susceptibility estimates’ comprised clearly inappropriate combinations, e.g. urinary coliforms were rarely tested for penicillin susceptibility. For 323 combinations, the proportion tested was >=90%. These ‘non-selective susceptibility estimates’ comprised clearly appropriate combinations, e.g. most pneumococcal isolates were tested for penicillin susceptibility. The remaining 296 combinations comprised ‘selective susceptibility estimates’ with proportions tested <90%. Details for all combinations and the full analyses are available from the authors; examples where effects were due to a single policy are shown in Table 1.


View this table:
[in this window]
[in a new window]
 
Table 1. Examples of selective susceptibility estimates from single laboratories practising the various selective testing policies compared with pooled non-selective all-Wales estimates
 
Inter-laboratory agreement for routine antibiotic sets was greatest for urinary coliforms and least for S. aureus. Initial attempts to analyse S. aureus as a single organism group were discarded when it became apparent that testing policies for MRSA and MSSA were frequently distinct. Some variation in testing was more apparent than real, e.g. substitution of norfloxacin for ciprofloxacin in the urinary coliform set. However, there was considerable further inter-laboratory variation in routine antibiotic sets and testing policies. Note that for some selective estimates, more than one policy contributed to selective testing; the examples in the text, tables and figure exclude these multiple effects.

Site-specific testing. For the 82 instances of selective estimates involving site-specific testing, susceptibility estimates tended to differ from pooled non-selective estimates when isolates from topically treated sites (eye, ear) were selectively tested or omitted. All were readily recognized from comparison of specimen site distributions for tested and untested isolates. As an example, the pooled non-selective estimate for ciprofloxacin resistance in H. influenzae (classing intermediate results as resistant) was 0.2%, whereas a laboratory that selectively tested eye and ear isolates, with a PT of 29%, yielded a resistance estimate of 1.8%. Statistical analysis suggested that this difference was unlikely to be due to chance (difference –1.6%, 95% CI –4.3% to –0.5%). Site-specific testing gave a large range of values for the proportion tested (mean 59%, range 0.8–89.5%), but differences from non-selective estimates were rarely more than two-fold.

Direct susceptibilities. These were involved in 106 instances of selective testing. The percentage tested was rarely >10%, and the instances invariably involved S. aureus from urine.

Second-line testing. These accounted for six instances of selective testing; all examples involved urinary coliforms, and cefuroxime or co-amoxiclav. The pooled non-selective estimate of co-amoxiclav resistance was 9.1%, compared to 70.3% for a laboratory with second-line testing (PT = 4.2%). The mean PT was 2% (range 0.6–4.2%), and a strong bias to high resistance was evident, with resistance estimates about 7- to 13-fold greater than non-selective estimates. Laboratories appeared to apply the policy with varying stringency, yielding a wide range of PT values, and a wide spread of selective resistance estimates. For cefuroxime, although no laboratory had tested more than 90% of isolates throughout the trial, one had tested at this level for approximately 4 years, and resistance was 5.1% for this period. This figure was taken as the best available non-selective estimate of urinary coliform resistance to cefuroxime.

Prescribing-specific testing. These accounted for 12 instances of selective testing. For ciprofloxacin, the pooled non-selective estimate for urinary coliforms was 1.8%, compared to 9.4% for prescribing-specific selective testing (PT = 1.3%) at one laboratory. Small numbers of isolates were involved in most instances; the percentage tested was <10%; resistance estimates were often high, more than three-fold greater than non-selective estimates in some instances; and results were sometimes rule-based rather than test-based.

Matched-susceptibility reporting. Of the nine instances of matched-susceptibility reporting, most involved penicillin-susceptible S. pneumoniae, isolated in mixed culture with H. influenzae, being reported as susceptible to ampicillin and/or co-amoxiclav. Equally, S. pyogenes isolated in mixed culture with penicillin-resistant MSSA were reported as flucloxacillin-susceptible by some laboratories. The PT was usually 10–20%, and deviation of more than two-fold from non-selective estimates was rare. Again, results were rule-based for some instances and test-based in others.

Test policy changes. Policy changes probably represent the most difficult area of selective testing, as the effects on susceptibility estimates depended on both the nature of the policy change and the date at which the change occurred relative to the period of data collection. The 81 instances were distributed between the species: 11 coliforms, 19 MRSA, 14 MSSA, 16 H. influenzae, 16 S. pneumoniae and five S. pyogenes.

Effect of selective testing on regional and local susceptibility estimates. All-Wales aggregate estimates that included or excluded contributions from laboratories with selective testing were compared (Tables 1 and 2). For most relevant species/antibiotic combinations, the aggregate proportion tested exceeded 50% and differences were minimal. However, for a few species/antibiotic combinations, the aggregate proportion tested was low because of widespread selective testing policies, and differences were large (e.g. for urinary coliform susceptibility to cefuroxime). In contrast, larger differences were found for estimates at Health Authority, Local Health Group or laboratory catchment level, where selective testing often made a large proportional contribution, and so tended to produce large perturbations.


View this table:
[in this window]
[in a new window]
 
Table 2. Some examples of the effects of selective testing policies on resistance estimates for Local Health Groups and Area Health Authorities compared to all-Wales pooled estimates from non-selective testinga
 
Percentage tested. The deviations of individual laboratories from all-Wales aggregate estimates tended to increase as the percentage tested fell (Figure 1). For this survey, if laboratory results with a proportion tested <20% were discarded, then variation of aggregate and individual laboratory estimates from pooled non-selective estimates tended to be small.



View larger version (14K):
[in this window]
[in a new window]
 
Figure 1. Examples of the various selective testing policies, comparing the proportion of isolates tested with the extent of deviation of the laboratory selective estimate from the pooled non-selective, non-rule-based all-Wales estimate. Negative values indicate increased resistance estimates from selective testing, compared to the all-Wales estimate. Error bars indicate 95% confidence intervals. A change in scale of the horizontal (proportion-tested) axis occurs at 20% to allow resolution of points representing severely selective testing policies. Open diamonds, site-specific testing; filled triangles, second-line testing; filled circles, direct susceptibility testing; filled squares, prescribing-specific reporting; open triangles, matched-susceptibility reporting.

 
Rule-based reporting

Five of the 14 laboratories incorporated rule-based reporting for H. influenzae susceptibility to erythromycin as intermediate or resistant, irrespective of laboratory test results. For the remaining nine laboratories, which reported test-based results, resistance estimates varied between 10.0% and 48.3%. Other instances were rule-based reporting of ciprofloxacin resistance in MSSA, S. pneumoniae and S. pyogenes where rule-based resistance estimates were 100%, compared to examples of individual laboratory test-based estimates of, respectively: 14.1% (95% CI 13.4–14.7%; PT = 95.4%); 10.6% (8.7–12.9%; PT = 77.9%); and 4.5% (3.5–5.7%; PT = 74.2%). These policies were distinct in their effects from others discussed above, as they affected non-selective estimates, as well as selective estimates (see, for example, matched susceptibilities above). For those laboratories applying rule-based reporting, non-selective estimates for the organism/antibiotic combinations mentioned above were close to 100% resistance. It was considered that no valid comparison of susceptibilities could be made between rule-based and test-based results.

Identification of urinary coliform isolates

Seven of the 14 laboratories reported urinary lactose-fermenting Gram-negative bacilli as ‘coliform’ or ‘LF’ almost exclusively. Three laboratories reported a proportion of urinary isolates (99.6%, 54.3% and 17.4%, respectively) as E. coli. Two laboratories ceased to report community urinary coliforms as E. coli during the study (originally reporting 32.6% and 82.9% E. coli, respectively) and two laboratories began to report urinary isolates as E. coli (65.7% and 95.7%, respectively). For two of the laboratories, identification was based on an indole test at minimum, one identified mainly by colony morphology, and one sometimes identified by morphology and sometimes biochemically. Urinary isolates reported as E. coli were significantly less resistant than those that were reported as coliforms for all antibiotics tested (Table 3).


View this table:
[in this window]
[in a new window]
 
Table 3. Resistance estimates for urinary isolates reported as E. coli compared with those reported as ‘coliform’
 

    Discussion
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
There is general agreement about the need for surveillance of antimicrobial resistance, but a lack of clarity in objectives.5 Identifying the target audience for surveillance data helps to define specific objectives: clinicians require information at a local level to make appropriate choices of antibiotics for empirical therapy; academics require information to identify and investigate new mechanisms of resistance; NHS organizations and other statutory bodies with health protection responsibilities require information to identify, investigate and control outbreaks; and, the UK Health Departments require information at a national level to determine policy priorities.6 The validity of routinely collected data for surveillance is dependent on the specific objective for which these data are used.

The results indicate that details of a laboratory’s policies on antibiotic testing can affect estimates of resistance based on routine diagnostic data. Selective policies often gave estimates that differed markedly from those based on non-selective testing in the same laboratory (where a policy change had occurred), other laboratories and pooled non-selective all-Wales figures. Second-line testing was the policy with the greatest impact on resistance estimates, similar but lesser impacts arose from other selective testing policies. Rule-based susceptibility estimates for erythromycin resistance in H. influenzae and ciprofloxacin resistance in Gram-positive cocci diverged markedly from test-based estimates. Smaller, but significant differences were found in susceptibility estimates for undifferentiated urinary coliforms and urinary isolates identified as E. coli.

For local surveillance (laboratory catchment, Local Health Group and Area Health Authority), the effects of selective testing were often large but at the all-Wales level, the effects were usually small. Selective testing inherently involves reduced isolate numbers and non-selective testing results from other laboratories often swamped the effect of selective testing. However, if regional standardization of policies was to occur, similar selective testing effects would be present in data from all laboratories and might remain unnoticed. With the current trend to inter-laboratory standardization, the all-Wales community resistance dataset may represent the last opportunity in the UK to analyse policy effects across a widely varied mix, and to untangle their effects on surveillance.

It is not our intention to suggest that any of these policies are correct or incorrect, or to suggest that policies should be changed to cater for surveillance of resistance. The primary purpose of a laboratory is to report bacteriology findings as accurately and helpfully as possible for individual patients, and this must not be compromised for any secondary purpose such as surveillance. Our intention was to document the effects of policies on local and regional surveillance and to determine whether any problems that they introduce into surveillance at local and regional levels can be detected and eliminated.

Testing policies are not the only source of aberrant data, and methodology effects must be considered. Current BSAC guidelines for urinary isolates of S. aureus recommend testing with systemic strength discs,7 so direct urine susceptibility results for these organisms would probably be discarded now. Resistance estimates for urinary and systemic isolates cannot be validly pooled or compared where they are based on distinct disc strengths or breakpoints. Aberrant non-selective estimates were mostly for one antibiotic (trimethoprim), affecting susceptibility estimates for Gram-positive cocci and results from three laboratories. This suggests methodological problems, which must be considered in analysis, but are a distinct issue from policy effects.

Economic and staffing problems may have constrained laboratories to selective testing policies, particularly in the case of site-specific testing. Susceptibility tests on a single plate, with a single set of discs covering both hospital and community isolates, have clear advantages in cost-constrained, understaffed laboratories. Pressure for selective testing may well have increased since the study, especially in laboratories that previously utilized eight discs per plate, with the change to the six-disc per plate BSAC protocol.7 Decisions on selective testing policies involve a host of factors. Testing a limited antibiotic set has advantages in laboratory economics, staffing and skill mix. Testing a broader set may have advantages in patient treatment and, from the results of this study, might well yield surveillance that is more comprehensive and reliable. UK laboratories test far fewer antibiotics per isolate than many other Western nations, but it is unlikely that the advantages to surveillance will alter this situation without financial input. However, we note the advantages of commercial automated systems that test large antibiotic sets and carry out biochemical identification, and that the effect of selective testing on surveillance should be considered when testing policies are reviewed.

Rule-based reporting brings a different class of problem, less amenable to simple detection. We consider that no valid overall resistance estimate can be generated from mixed majority rule- and test-based estimates (exemplified here for erythromycin-resistance in H. influenzae, and ciprofloxacin-resistance in Gram-positive cocci). The estimates derive from fundamentally distinct beliefs and methodology on prediction of clinical efficacy in these instances. There are no easy solutions here, other than for microbiologists to adopt a consensus view and practice based on clinical evidence in each specific case where the problem occurs. Detection of rule-based reporting relied on thorough familiarity with the testing policies of the contributing laboratories. A strategy of reporting rule-based susceptibilities as comments, rather than results would avoid confusion of surveillance data, and could probably be implemented with the automated comment functions available on many modern laboratory information systems.

The effects of regarding heterogeneous groups of organisms as a single class were clear in the analyses of urinary coliforms identified as E. coli. Species-level identification is becoming a prerequisite for interpretation of susceptibility tests for disc sensitivities in the BSAC scheme7 and in many automated systems. This has advantages for surveillance, and for individual patients, where identification of a urinary isolate as a Gram-negative bacillus other than E. coli carries implications about the underlying pathology. However, failure to identify organisms to species level will have little effect on empirical antibiotic selection for the treatment of urinary tract infection and is only an issue for surveillance when comparing areas with different practices or when practice in one area changes over time.

We suggest that laboratories should keep a log of changes in rule-bases, testing policies and methods, specifying the nature of the change, the antibiotics and species affected and the date of change. This could save considerable time in tracing discordant data during local surveillance analysis, and might well provide useful information for those involved in regional and broader levels of surveillance.

Figure 1 clearly illustrates the progressively greater spread of resistance estimates as the proportion of isolates tested by a laboratory decreases, with a concomitant increase in the range of the 95% confidence limits, and a clear trend to higher resistance estimates for more stringently selective testing. There are good theoretical reasons to suppose that many policies with stringent isolate selection (e.g. second-line testing, prescribing-specific testing) may be biased towards selective testing of resistant isolates, and the results outlined here confirm this. Further, as the proportion of isolates tested decreases, the number of isolates contributing to the selective estimate falls, and uncertainty due to potential chance sampling variation increases. There is no clear break in this progression, but examination of the dataset as presented in Figure 1 suggests that resistance estimates derived for a laboratory testing less than about 20% of isolates for the species/antibiotic combination should be treated with profound suspicion. The exception to this would be large surveys with a formally randomized isolate testing design.

Where the proportion of isolates tested by a laboratory ranges from about 20% to 90%, further detailed analysis may sometimes indicate that a portion of the data is valid. As an example, several instances of policy changes from minimal to non-selective testing were found in this dataset, allowing resistance estimates to be calculated over the period of non-selective testing. Resistance estimates where the proportion of isolates tested is >=90% appear unlikely to be affected by selective testing, but need to be assessed for other effects, e.g. laboratory methodology problems, before being accepted as evidence of real inter-laboratory variation.

This triage is based on arbitrary break values for the proportion tested, and is subject to confirmation in further studies, but may be useful to others who wish to direct limited analysis resources into those areas that are likely to yield the greatest return. Note that the proportion tested can only be applied in this way for data from a single laboratory on a single antibiotic/species (or species subgroup) combination, where the laboratory policy is uniform for all isolates of the species. It would make little sense to triage data based on the proportion tested for pooled data from several laboratories with distinct policies, or for pooled species subgroup data where policies differ between the subgroups—as encountered here with MSSA and MRSA.

Overall, the study has shown that the intricacies of laboratory testing policies do affect estimates of community resistance, but that many of the resulting problems can be detected during initial data processing. Surveillance provides a context for local figures by allowing comparison with broad-based resistance data. This can be used to detect incipient problems (atypical trends, or in quality control), to inform decisions on antibiotic usage, and to monitor the effects of changes in practice. These functions are important, particularly in the context of the worrying worldwide trend to increased resistance, and rely heavily on confidence in surveillance data. It may be that many putative biasing effects are illusory, or sufficiently consistent to allow correction, or can be eliminated by application of simple rules. Resolution of the reality from the mythology of bias in resistance estimates is an important step in instituting reliable surveillance.


    Acknowledgements
 
We thank Dr D. M. Livermore for constructive comments on the manuscript and the staff of the 14 laboratories involved whose help has been invaluable; the three Welsh border laboratories (Chester, Hereford and Shrewsbury Microbiology Laboratories) that are contributing data to the project, and the Wales Office of Research and Development for Health and Social Care for their financial support of the project.

The Welsh Antibiotic Study Group comprises: Dr K. Al Shafi, Royal Gwent Hospital, Newport; Dr C. Banerjee, Princess of Wales Hospital, Bridgend; Dr N. J. B. Carbarns, Nevill Hall Hospital, Abergavenny; Dr C. Cefai, Wrexham Maelor Hospital; Dr K. El-Bouri, Prince Charles Hospital, Merthyr Tydfil; Dr K. A. Fitzgerald, National Public Health Service for Wales, Cardiff; Dr G. Harrison, Prince Phillip Hospital, NPHS for Wales, Microbiology Carmarthenshire; Dr D. N. Looker, NPHS, Microbiology Rhyl; Dr L. Macfarlane, NPHS for Wales, Microbiology Aberystwyth; Dr P. Thomas, NPHS for Wales, Microbiology Swansea; Dr R. Salmon, CDSC Wales; Dr M. Sheppard, Withybush Hospital, Haverfordwest; Dr M. D. Simmons, Welsh Assembly; Dr M. Walker, NPHS for Wales, Microbiology Bangor; Dr D. White, Royal Glamorgan Hospital, Pontypridd; and the authors of this paper.


    Footnotes
 
* Corresponding author. Tel: +44-29-2052-1997; Fax: +44-29-2052-1987; E-mail: Brendan.mason{at}nphs.wales.nhs.uk Back

§ Members of the Welsh Antibiotic Study Group are listed in the Acknowledgements. Back


    References
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
1 . Oppenheim, B. A. (2002). Surveillance of antibiotic resistance in urinary tract infections—are we misleading prescribers? Communicable Disease and Public Health 5, 181–2.[Medline]

2 . Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association 22, 209–12.

3 . Newcombe, R. G. (1998). Two-sided confidence intervals for the single proportion: comparison of seven methods. Statistics in Medicine 17, 857–72.[CrossRef][ISI][Medline]

4 . Newcombe, R. G. (1998). Interval estimation for the difference between independent proportions: comparison of eleven methods. Statistics in Medicine 17, 873–90.[CrossRef][ISI][Medline]

5 . Lewis, D. (2002). Antimicrobial resistance surveillance: methods will depend on objectives. Journal of Antimicrobial Chemotherapy 49, 3–5.[Abstract/Free Full Text]

6 . Hunter, P. A. & Reeves, D. S. (2002). The current status of surveillance of resistance to antimicrobial agents: report on a meeting. Journal of Antimicrobial Chemotherapy 49, 17–23.[Abstract/Free Full Text]

7 . Andrews, J. for the BSAC Working Party on Susceptibility Testing. (2001). BSAC standardized disc susceptibility testing method. Journal of Antimicrobial Chemotherapy 48, Suppl. S1, S43–57.[CrossRef]