Department of Economics, Strathclyde University, Curran Building, 100 Cathedral Street, Glasgow G4 0LN, UK
Received 14 May 2002; in revised form 15 May 2002;
ABSTRACT
The Swedish State Health Technology Board (SBU) has published a Report on the treatment of alcohol and drug misuse (SBU, 2001). This article is a brief Commentary on the economic issues raised in Chapter 9 of the Report, in particular, the question of how cost-effective are the different treatment alternatives? An outline is given of how the authors approached the economic work, with particular reference to a standard checklist approach to judging the quality of published economic appraisals. A paucity of such appraisals was retrieved and detailed review of just 16 papers was undertaken. The authors are critical of the lack of quality of economic work in the substance misuse area and their main points are summarized here. The main conclusion drawn is that no economic judgements can be made about alternative treatment approaches, and more study and research are needed in this area. Chapter 9 is, on balance, a good attempt at a critical review of the economic appraisal literature. Unfortunately the main Report provides a series of summary judgements on the effectiveness or otherwise of alternative treatments, which, while recognizing the need for further cost-effectiveness work, essentially ignores the conclusion drawn in the economics chapter. This is likely to do a disservice to the cause of appropriate resource allocation in the substance misuse area.
INTRODUCTION
The Swedish State Health Technology Board (SBU) has published a Report on the treatment of alcohol and drug misuse (SBU, 2001). This article is a brief Commentary on the economic issues raised in Chapter 9 of the Report, entitled Economic evaluation of the substance abuse care sector an overview. One of the fundamental questions raised in the Report is that of how cost-effective are the different treatment alternatives?
METHODS USED
The authors of the SBU (2001) Report appear to have conducted their economic literature search on sound principles. Initially 1200 studies were identified, but most were not economic assessments (which requires the comparison of inputs and outcomes, suitably measured and valued in cost and benefit terms). Twenty-four studies contained formal economic analysis, eight of which were randomized clinical trials.
The SBU (2001) Reports authors followed what is now considered good practice in using the guidelines for economic appraisal which are familiar to economists, and are used by the British Medical Journal (Drummond and Jefferson, 1996) and other medical journals. Summary Tables covering 24 studies employing a variety of treatment methods are provided. A 10-point checklist was applied (Drummond et al., 1987
, 1997
), which considered, inter alia, the study question; the identification, measurement, and valuation of costs and effects; the timing of projects; and issues of uncertainty. Sixteen studies were deemed suitable to be assessed for quality according to the checklist, and a further Table summarizes the checklist results. Just two studies were judged to be of high quality, whereas seven others were considered of medium, and seven more of low, quality. A very detailed appendix devotes
3 pages to each of these 16 studies and applies the checklist to them all. There was no attempt at meta-analysis of the economic studies reported. A useful annex explains alternative methods of economic appraisal.
Hence the Report makes a creditable attempt to apply detailed economic criteria to the studies reviewed. Two issues can be raised. First, the basis for making the overall quality judgements is not made explicit (e.g. one of the high quality studies scores yes for eight out of 10 on the checklist, the other six yes and two yes/no). Second, as an author who has suffered from reviewers applying checklists developed in the late 1980s to a paper written in the late 1960s, I can sympathize with the authors of the five studies published before the Drummond et al. (1987) book being subjected to such ex post quality judgements (three low and two medium)!
RESULTS OBTAINED
The last three paragraphs of Chapter 9 of the SBU (2001) Report helpfully summarize the authors judgements on the material they reviewed. Key points made are: (1) the generalizability of the results is doubtful, since inferences can only be drawn in many cases for the populations investigated; (2) studies are often based on far too few individuals, with very short follow-up times, in most studies less than 1 year; (3) most studies are of imperfect quality with respect to the measurement and valuation of costs and treatment consequences; (4) international studies are few and documentation on different technologies and strategies costs and effects is insufficient; (5) the implications of the studies for the situation in Sweden are few: economic evaluations are required which bear upon Swedish questions, Swedish conditions, and Swedish data.
It was also noted that recorded clinical studies used different effect measures, which renders more difficult comparisons between different resource uses. There is thus a need to develop common effect measures which can be used for judgements of the relative usefulness of the different goals of the substance abuse care sector.
The authors of the SBU (2001) Report point out that this sector makes use of significant resources, yet no available studies can be found that show how effectively these resources are used.
It is concluded that it is important to undertake economic evaluations of the Swedish care sector, its scope and aims, and that the leaders of the sector who answer for the use of resources which the taxpayers have entrusted to them to use effectively should take the initiative for such studies.
DISCUSSION AND COMMENTS
In summary, the literature reviewed in Chapter 9 of the SBU (2001) Report allows no economic judgements to be made about the alternative treatment approaches, and we need more study and research in this area, to which I can point to a review conducted in Scotland (Ludbrook et al., 2001), which broadly comes to the same conclusions, and can only concur.
It is stated in the English summary of the SBU (2001) Report that care for substance abuse can be improved by (a) shifting resources away from ineffective treatment methods and into treatment methods that have been documented as effective and (b) committing more resources to treatment programmes that apply evidence-based methods.
Whilst I do not cavil at the latter two points, it should be pointed out that the Report makes a series of judgements about treatment and care. When it comes to these judgements, the economic aspects are simply ignored, apart from recognizing the need for more cost-effectiveness studies. What is the point of doing a serious review of the economic literature if no real connection is to be made? If this seems harsh, it can be observed that the Reports summary concludes with a nice clear table with headings is treatment effective and which therapies are best showing the effectiveness of some interventions, and the lack of effectiveness of other interventions, with an accompanying text which implies that these judgements are scientifically sanctioned. Although the statement the scientific evidence is too weak or contradictory to permit conclusions about the cost-effectiveness of different treatment methods appears in the section just above this Table, a serious disservice will be done to optimal resource allocation in Sweden (and elsewhere) if decisions are made on the advice of this Table without considering the economic implications.
What is needed in this area is an evaluation culture which would allow alternative treatments and strategies for substance misuse to be monitored and evaluated, including economic evaluation. This is not a plea simply for (often costly) research projects, although policy in this area undoubtedly needs to be informed by more research and of higher quality, as this Report amply demonstrates. Health economists would be delighted if both baseline and ongoing data on the inputs (costs) and effects of alternative therapies were to be collected on a regular basis, thus permitting economic appraisal. However, it is not being advocated that decisions be made purely on economic grounds, simply that decision-making ought to benefit from the presence of economic information, rather than its absence.
REFERENCES
Drummond, M. and Jefferson, T. (1996) Guidelines for authors and peer reviewers of economic submissions to the BMJ. British Medical Journal 313, 275283.
Drummond, M., Stoddart, G. and Torrance, G. eds (1987) Methods for the Economic Evaluation of Health Care Programmes. Oxford University Press, Oxford.
Drummond, M., OBrien, B., Stodddart, G. and Torrance, G. eds (1997) Methods for the Economic Evaluation of Health Care Programmes, 2nd edn. Oxford University Press, Oxford.
Ludbrook, A., Godfrey, C., Wyness, L., Parrott, S., Haw, S., Napper, M. and van Teijlingen, E. (2001) Effective and Cost-Effective Measures to Reduce Alcohol Misuse in Scotland: A Literature Review. Scottish Executive, Edinburgh.
SBU (2001) Behandling av alkohol-och narkotikaproblem. En evidensbaserad kunskapssammanställning. Statens beredning för medicinsk utvärdering, rapport nr 156. Stockholm. ISBN 91-87890-73-9.