Questionnaire standards in epidemiology

John Gallacher

MRC Epideiology Unit (South Wales), Llandough Hospital, Penarth, South Glamorgan, CF64 2XW, UK.

Sir—The call for improved standards in questionnaire design in epidemiology1 is long overdue. The proposed clearing house, committee or working group would likely improve standards. The further proposal of setting up workshops or task forces to address specific issues may also promote better practice. The risk attached to institutionalizing reference points for good practice, however, is that the virtues of validity and comparability become the pressure for conformity.

Easier access to questionnaires is welcome. The benefit of a clearing house, however, is not a straightforward issue of facilitating research. Questionnaires and their findings can have clinical and legal, and increasingly, commercial significance. Unless the contents of a clearing house were limited to research tools of unknown validity and significance it is inconceivable that rules and conditions of access would not be required.

Gathering experts to administrate a resource could define a committee. The responsible administration of questionnaires has much to commend it. Professional bodies in Psychology have practised it for many years. In Epidemiology, however, the potential for questionnaire use is so broad, that quite apart from encroaching on the expertise of other disciplines, the task is unworkable. A way forward might be to include only those questionnaires used for medical diagnosis. Sadly, this would defeat much of the object of a general improvement in questionnaire use throughout the discipline.

We all applaud the standardization of procedures, but can a working group achieve this? In fact, this is the heart of the problem. How can a working group set standards for best practice given the range and variety of questionnaire use in Epidemiology? Standardization may be a necessary part of an ongoing process of reducing uncertainty in data, but the real ‘standard’, of course, is comparability of findings. The critical parameters of questionnaire use, those which provide comparable findings, are a matter for research not opinion. Appropriate levels of standardization in any area of research will only come through the comparison of results from a known range of varied practice; not from the deliberations of a working group.

At one level, any vehicle which improves the quality of questionnaire data in epidemiology generally, is welcome. It is worth noting, however, that members of the scientific community have a vested interest in improving standards once the issues are understood and publicly acknowledged. Rather than institutionalize the process of improving questionnaire use, perhaps a more fundamental approach is to stimulate the quality of public debate and understanding, in order to improve the judgement of the researcher.2,3 A task which your leading article attempted admirably.

References

1 Olsen J. Epidemiology deserves better questionnaires. Int J Epidemiol 1998;27: 935.[Free Full Text]

2 Gallacher J, Davey Smith G. A framework for the adaptation of psychological questionnaires for epidemiological use: An example of the Bortner Type A scale. Psychol Med 1989;19:709–17.[ISI][Medline]

3 Eaker S, Bergstrom A, Adami HO, Nyren O. Response rate to mailed epidemiologic questionnaires: A population-based randomized controlled trial of variation in design and mailing routine. Am J Epidemiol. 1998;147:74–82.[Abstract]


 

Authors' response

Jørn Olsen

DK-8000 Aarhus C, Denmark. Tel: +45 8942 6102; Fax: +45 8613 1580; E-mail: rj{at}soci.au.dk

Sir—Our aim was to start a debate and to encourage an appropriate body to act. Gallacher's letter is therefore most appreciated.

The starting point is that much good work in developing questionnaires is wasted. Too often, mistakes are repeated and we lack agreed standards for validating questionnaires. Existing questionnaires are not always readily available and the protocol for their use is not known or observed. In part, these problems are related to the fact that there is no organizational protocol for the development, validation, archiving and licensing of questionnaires. We believe that a committee or group (the name is unimportant) is needed to explore these issues further, and provide advice to the wider academic community with regard to the design, management and reuse of instruments. The amount of work that a single committee can undertake is obviously limited, but discussion around these issues would be a useful first step.

As a way forward, we have suggested the value of a few select questionnaires being subject to scrutiny with regard to their robustness in different settings, and their validity in generating good epidemiological data. We suggest that a peer group might develop guidelines for others to follow in designing and conducting research, for example with respect to adherence to protocol in the use of validated instruments. Third, a body (e.g. a designated university) could set up an archive of questionnaires which have wide acceptance as ‘gold standard’ instruments.

In making these suggestions, we are seeking improved quality of research, not seeking to limit creativity or productivity. Indeed, we would hope that the processes we suggest would stimulate further work rather than stifle criticism. We also hope our paper will encourage funding bodies to give higher priority to this area of research.





This Article
Extract
FREE Full Text (PDF)
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Request Permissions
Google Scholar
Articles by Gallacher, J.
Articles by Olsen, J.
PubMed
PubMed Citation
Articles by Gallacher, J.
Articles by Olsen, J.