1 Department of Pharmacology, Therapeutics & Toxicology, University of Wales College of Medicine, Heath Park, Cardiff CF14 4XN; 2 Bristol Centre for Antimicrobial Research and Evaluation, Department of Microbiology, Southmead Hospital, Westbury-on-Trym, Bristol, UK
Received 19 December 2002; returned 18 January 2003; revised 17 February 2003; accepted 23 April 2003
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Methods: Data from the measurement of 40 external quality assessment samples from 358 laboratories providing a therapeutic drug monitoring service were analysed.
Results: Significant differences between techniques in accuracy and precision were observed for all drugs. Coefficients of variation ranged from 4.1% to 9.8% for the aminoglycosides and from 6.7% to 11.7% for vancomycin. The percentage difference in measurements from the weighed-in drug concentration ranged from 10.1% to +4.0% for the aminoglycosides and from 3.5% to +5.7% for vancomycin. The Dade Behring Emit immunoassay was notable in producing significantly more outliers (>4 S.D. from the weighed-in concentration) than other techniques in the measurement of gentamicin, amikacin and vancomycin.
Conclusions: All assays performed to a satisfactory standard for measurement in non-renal patients, but none met the more stringent standards desirable for monitoring patients with renal impairment.
Keywords: gentamicin, tobramycin, amikacin, netilmicin, immunoassays
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
For all laboratories providing clinical assays for vancomycin and the aminoglycosides in the UK (and in parts of Europe and the USA), participation in an external quality assessment (EQA) scheme is mandatory. This is not only to fulfil the standards set by the Clinical Pathology Accreditation agency, but also so that individual laboratories can monitor their assay performance against that of their peers. In 2002, there were over 350 laboratories receiving monthly EQA samples from the United Kingdom National External Quality Assessment Scheme (UKNEQAS) for antibiotic assays (Bristol, UK), in collaboration with the Heathcontrol EQA scheme (Cardiff, UK). In this study, we report our findings following a retrospective analysis of EQA data to assess the performance, in terms of accuracy and precision, of assays for vancomycin and the aminoglycosides.
![]() |
Materials and methods |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Samples for analysis were prepared from pooled human serum (National Blood Service, Bristol) negative for HIV antibody, hepatitis C antibody and hepatitis B surface antigen. Gentamicin, tobramycin, amikacin (Sigma Chemical Co., Poole, UK), netilmicin (Schering Plough, Welwyn Garden City, UK) or vancomycin (Eli Lilly, Indianapolis, IN, USA) were added to serum accurately by weight, to give clinically relevant concentrations from subtherapeutic to toxic. Serum was mixed thoroughly and dispensed as 0.5 to 1 mL aliquots. The serum samples were not frozen after antibiotic spiking, no preservative was added, and they were distributed monthly, by post or courier, within 1 or 2 days of manufacture to EQA scheme members. All antibiotics were distributed as single analyte samples, apart from gentamicin and vancomycin, which were distributed in combination for the last 14 months. Laboratories were requested to report, within a 23 week deadline, the antibiotic concentration measured using their routine protocol and the assay technique. Data from the samples distributed in the 40 months between January 1999April 2002 were included in the statistical analysis.
Data screening
Results for each analyte were first screened to remove transcription and misclassification errors. Measurements >4 S.D. from the weighed-in concentration were rejected using the robust method of Healy.5 Significant (P < 0.05) differences between immunoassays in the frequency of rejected measurements were identified by 2 tests.
Assessment of accuracy and precision
Non-rejected data for each analyte were classified according to technique, and the mean and S.D. calculated for the subgroups. Data from techniques providing a total of <70 measurements for an analyte were omitted from the analysis.
The difference from the weighed-in concentration in the all-technique mean result for each analyte showed an increasing variability with increasing drug concentration. The difference from the weighed-in concentration in the individual-technique subgroup mean, expressed as a percentage, was a more stable parameter with respect to drug concentration and was therefore selected to assess technique accuracy.
The S.D. of measurements increased with increasing drug concentration. Expressing the S.D. as a percentage coefficient of variation (%CV) relative to the all-technique mean (S.D./mean x 100) gave a parameter more uniform with respect to drug concentration, and %CV was therefore chosen to assess technique precision. However, %CVs increase towards infinity as drug concentrations approach zero. For samples containing the lowest drug concentrations, %CVs were large compared with other data, and eight data sets with %CVs from 1332 were omitted from the analysis (gentamicin: 0.3 mg/L; tobramycin: 0.2 and 0.3 mg/L; amikacin: 2.0 and 2.5 mg/L; netilmicin: 0.7 mg/L and vancomycin: 3.8 and 4.4 mg/L). In addition, there were comparable large errors in the weighed-in concentration calculated for nine samples, and these data were also omitted from the analysis. The number of samples included in the analyses were gentamicin 38, tobramycin 36, amikacin 38, netilmicin 36 and vancomycin 35. Techniques were compared by one-way analysis of variance, and significant differences between means identified by the StudentNewmanKeuls test (P < 0.05).
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
|
|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The popular Abbott assays were in the statistical group with the highest precision for four out of five drugs (exceeded only by the Roche FPIA for tobramycin). The AxSYM assays were slightly superior in precision to those on the TDx, reaching significance for netilmicin. The AxSYM assays were also more precise than Sigma FPIA assays for gentamicin and amikacin, and the Biostat FPIA for amikacin. The CEDIA, turbidimetric and chemiluminescent technologies were of comparable precision to the various FPIA assays for gentamicin. In contrast, the Dade Behring Emit assay differed significantly from the other techniques, producing significantly higher %CVs in non-rejected data for all four drugs for which sufficient data were available for analysis, and significantly more rejected measurements (>4 S.D. from the weighed-in concentration) for three of the drugs.
There was a difference in accuracy of techniques between the aminoglycosides and vancomycin. For the aminoglycosides, there was a general tendency for techniques to underestimate drug concentration relative to the weighed-in concentration. The latter is a gravimetric estimate from a single laboratory and subject to error. The observed differences in accuracy between techniques are thus to be interpreted in a relative manner. The Dade Behring Emit, Beckman turbidimetric, Biostat FPIA and Sigma FPIA assays produced results closest to the weighed-in value. The Abbott AxSYM, Abbott TDx and Bayer turbidimetric assays produced significantly lower measurements, whereas the Bayer chemiluminescence and CEDIA assays were significantly lower again for gentamicin. The Roche FPIA was unusual in showing biases that differed between analytes. It gave significantly higher measurements for gentamicin and significantly lower values for amikacin than any other technique. The latter negative bias may be responsible for the significantly greater number of rejected measurements identified relative to the weighed-in value for the Roche amikacin FPIA.
For analysis of vancomycin, all methods, with the exception of Abbott AxSYM, gave a positive bias relative to the weighed-in value, with Dade Behring being numerically closest to the weighed-in value. The negative bias of the AxSYM technique differed significantly from all other methods. The explanation for this variation lies in the differences between assays in their cross-reactivity to an inactive crystalline degradation product of vancomycin (CDP-1) produced by spontaneous degradation in samples and found in high concentrations in patients with renal failure.6,7 Assays other than the AxSYM technique are based on polyclonal antibodies that cross-react, to varying degrees, with CDP-1, whereas the AxSYM system uses a murine monoclonal antibody with less cross-reactivity. The AxSYM assay will thus report lower, and potentially more accurate, estimates of vancomycin depending on the CDP-1 content of samples.
Although statistically significant differences were identified between techniques, their scale relative to clinical needs should be considered. Desirable standards of performance can be estimated from the time interval between doses and the average elimination half-life for a drug.8 For a typical aminoglycoside with a 3 h half-life and dosing every 8 h, a %CV of 18 is a realistic analytical goal. This level was achieved by all techniques where the total error resulting from the combination of imprecision and inaccuracy varied between 9%16%. However, if renal impairment were to increase half-life to 24 h, the desirable %CV falls to 3, and no available technique met this criterion. For vancomycin, a half-life value of 6 h, and dosing every 12 h, equates to a %CV of 15 as a desirable goal. All available techniques met this standard, with an observed total error varying between techniques by 10%14%. Again, reduced renal clearance will increase the desirable standard of performance beyond that currently achieved. The increase in dose interval that would result from the use of once-daily dosing raises the desirable CV, but not sufficiently to make current assays satisfactory for use in renal patients.
A more worrying aspect of performance was the fact that significant differences between techniques were detected in the number of measurements >4 S.D. from the weighed-in concentration. If these had all been the result of non-assay-dependent errors, such as transcription errors, between-technique differences would not have been expected. The poorer performance of the Dade Behring Emit assays in this comparison correlated with their poorer precision seen in analysis of non-rejected data, and suggests that Emit assays are more prone to producing occasional measurements with sufficient bias to have potential adverse clinical consequences.
In summary, all the commercial immunoassays reviewed by the study achieved the desirable standard of performance required for the provision of a routine therapeutic drug monitoring service for aminoglycosides and vancomycin in normal patients. In patients with renal failure, a higher standard of precision and accuracy is required and no currently available assay met the theoretical standard. Users of the Dade Behring Emit assays should monitor their assays, by means of quality control procedures, for occasional grossly discrepant results, and seek to identify and eliminate the cause.
![]() |
Acknowledgements |
---|
![]() |
Footnotes |
---|
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
2 . Reeves, D. S., MacGowan, A. P., Holt, H. A. et al. (1995). Therapeutic monitoring of antimicrobials: a summary of the information presented at the UK NEQAS for Antibiotic Assays meeting for participants, October, 1993. Journal of Antimicrobial Chemotherapy 35, 21326.[ISI][Medline]
3 . British Medical Association and Royal Pharmaceutical Society of Great Britain (2002). British National Formulary, Number 44 (October 2002). British Medical Association and Royal Pharmaceutical Society of Great Britain, London, UK.
4
.
Tobin, C. M., Darville J. M., Thomson A. H. et al. (2002). Vancomycin therapeutic drug monitoring (TDM): Is there a consensus view? The results of a UK National External Quality Assessment Scheme for Antibiotic Assays (UK NEQAS) questionnaire. Journal of Antimicrobial Chemotherapy 50, 7138.
5 . Healy, M. J. R. (1979). Outliers in clinical chemistry quality-control schemes. Clinical Chemistry 25, 7757.
6 . Somerville, A. L., Wright, D. H. & Rotschafer, J. C. (1999). Implications of vancomycin degradation products on therapeutic drug monitoring in patients with end-stage renal disease. Pharmacotherapy 19, 7027.[CrossRef][ISI][Medline]
7 . Trujillo, T. N., Sowinski, K. M., Venezia, R. A. et al. (1999). Vancomycin assay performance in patients with acute renal failure. Intensive Care Medicine 25, 12916.[CrossRef][ISI][Medline]
8
.
Fraser, C. G. (1987). Desirable standards of performance for therapeutic drug monitoring. Clinical Chemistry 33, 3879.