The validity of self-reported use of health care across socioeconomic strata: a comparison of survey and registration data

Sijmen A Reijnevelda and Karien Stronksb

a TNO Prevention and Health, Department of Public Health, Leiden, The Netherlands.
b Academic Medical Center, University of Amsterdam, Institute of Social Medicine, Amsterdam, The Netherlands.

Sijmen A Reijneveld, TNO Prevention and Health, Department of Public Health, PO Box 2215, 2301 CE Leiden, The Netherlands. E-mail: SA.Reijneveld{at}pg.tno.nl

Abstract

BackgroundSocioeconomic differences in health and in use of health care are well known. Most data on socioeconomic differences in health care utilization are based on retrospective self-report in community surveys, but the evidence on the validity of self-reported utilization of health care across socioeconomic groups is limited. The aim of this study was to assess the validity of self-reported utilization of health care across socioeconomic groups in the general population.

MethodsWe compared the concordance of self-reported and registered hospitalization (one year, n = 1277), and utilization of physiotherapy (one year, n = 1302) and use of prescription drugs (3 months, n = 899), by socioeconomic group (educational level, income, occupational status). Data came from a face-to-face health interview survey in Amsterdam and a health insurance register, and were limited to native Dutch and lower and middle income groups.

ResultsConcordance between reported and registered utilization was generally good to excellent; kappas (agreement adjusted for chance agreement) and percentage accurately reporting ranged from 0.60 and 80% (drugs) to 0.80 and 96% (hospitalization). They differed little, and without statistical significance, between people of low socioeconomic status and others. Assessment of socioeconomic groups in more detail yields somewhat more variation, but no systematic trend in concordance by higher socioeconomic status.

ConclusionSelf-report offers a reasonably valid estimate of differences in utilization of health care between socioeconomic groups in the general population, at least for lower and middle income groups.

Keywords Socioeconomic status, health care, health surveys, reproducibility of results, validation study

Accepted 6 June 2001

Socioeconomic (SE) differences in health1–7 and in use of health care8–19 are well known. In most countries people of low socioeconomic status (SES) have poorer health and higher mortality whereas their utilization of health care may be either higher or lower, depending on country and type of care. Van der Meer summarizes the available evidence on industrialized countries, which shows a clear distinction between the US and other countries such as Canada, Norway, the UK and the Netherlands.15 In the US, people of high SES have relatively more physician contacts, whereas people of low SES visit the emergency room more frequently. In the other countries, people of low SES visit primary care physicians relatively frequently, which is partly due to their poorer health status. As for more specialized care, people of low SES mostly use it less than others, especially when taking their (poorer) health status into account. Most recent data generally confirm this pattern,9,14,16,18 although in the UK emergency admissions also seem to occur relatively frequently among deprived people,17 and in some studies (in Canada and the Netherlands Antilles) health status more or less accounts for SE differences in general practitioner (GP) utilization.12,13,19

Most data on SE differences in health care utilization are based on retrospective self-report in community surveys,9,12,13,18,19 with a possible risk that SE differences in recall and reporting may affect comparisons of utilization across SE groups. Suppose, for instance, that the recall of people of low SES is poorer or that they consider their utilization to be socially undesirable. Both may result in underreporting. An actually higher utilization among low SES people would then remain hidden or an actually similar utilization across SE groups would result in a lower self-reported utilization among low SES groups.

Evidence on the validity of self-reported utilization of health care across SE groups is limited.18,20–24 The available studies mostly used predefined cohorts in which utilization had already been registered and self-reported use of health care was investigated retrospectively.20–24 The remainder are confined to specific groups or specific types of care.18,22,24,25 No studies are available on the validity of self-reported utilization of various types of health care across SE groups in the general population. This study therefore examines the validity by comparing data on utilization from a survey with data from registration in a general population sample and across several types of care.

Methods

This study is based on linkage of data from a previously performed survey and a health insurance fund (HIF) register, in Amsterdam, the Netherlands, and is restricted to respondents born in the Netherlands.

Subjects
The survey covered a random sample of the Amsterdam population register consisting of residents aged >=16 years, excluding people living in institutions. Registration in this population register is obligatory. The present study concerns respondents who were born in the Netherlands, could be linked to an HIF record and had HIF insurance for the entire study period (1992 and 1993) (Figure 1Go). They were personally asked by trained interviewers about their health and use of health care in the 3 or 12 months preceding the interview. Details of fieldwork,26–28 non-response,27,29 and linkage procedure29,31 have been reported previously.



View larger version (22K):
[in this window]
[in a new window]
 
Figure 1 Number of people excluded in each step of the procedure (original sample: 8335 people; response rates per group between brackets)

 
Respondents who had been born outside the Netherlands were excluded because a previous study showed that concordance between registration and self-report tends to be lower among non-native respondents. This may be explained by cultural differences and language problems.30 Inclusion of non-native respondents would therefore confound the present analysis on SE differences, especially as they mostly belong to the low SES group.27

As HIF insurance is obligatory for and limited to those with (relatively) lower incomes (for the whole of the Netherlands this applies to 63% of the population, for Amsterdam to 66%), the analyses are limited to the lower and middle income groups. There were only two reasons for insurance by a particular HIF to be stopped: movement out of its catchment area or an increase in income. The number of people excluded at each step as well as the associated response percentages are presented in Figure 1Go.

Data
The data on health care utilization concerned the types of care on which information could be extracted from the HIF register, were fully covered by it, and were quite common (prevalence >5%). Moreover, they had to be mentioned in the survey in the same way as they were registered by the HIF. (Overnight) hospitalization and use of (ambulatory) physiotherapy in the year preceding the interview met these criteria, as did the use of prescription drugs by the interviewed men in the 3 months preceding the interview. (Use of GP care was asked about in the survey but the number of contacts with GPs was not registered by the HIF, due to the fact that Dutch GPs receive a yearly remuneration for HIF insured patients irrespective of the number of visits). The questions included in the interview are shown in the Appendix. Information was extracted from the register data for the same period as covered by the interviews. However, even then time frames may differ somewhat. For instance, people may use prescription drugs at some time after they were provided.

For all three types of care, the analysis was limited to people who responded to the questions in the interview (in all cases: >98%); regarding registered data, no missing values occurred. The analyses regarding hospitalization and use of physiotherapy were restricted to people interviewed in 1993, as the information from the HIF did not cover 1991 (and that year was included if we looked back one year from 1992). Furthermore, with regard to hospitalization, women who had given birth to a child in the preceding year (n = 13) were excluded because short-term overnight stays in maternity clinics were not registered as hospitalizations but had been asked about as such in the survey. For prescription drugs, the analysis was restricted to men because oral contraceptives had not been asked about in the survey and could not always be identified as separate drugs in the HIF register. The remaining numbers of respondents for each type of care are presented in Figure 1Go and Table 1Go.


View this table:
[in this window]
[in a new window]
 
Table 1 Estimated utilization rates (95% CI) of hospitalization, physiotherapy and prescription drugs on the basis of registration data and of self-report and concordance between registration and self-report
 
Socioeconomic status was measured by its traditional indicators: income, occupational status and educational level.32,33 In most analyses, we dichotomized categories of each indicator by comparing the most unfavourable category with all others. In this way, we tried to maximize the power of our study to detect a lower validity of self-report among people of low SES (among whom we expected a lower validity to be most likely), and to limit the number of comparisons.

Income was defined as household income at four levels, adjusted for household size (single person equivalents: <Dfl 1400; Dfl 1400–1899; Dfl 1900–2749; >Dfl 2750); in most analyses this was dichotomized as: social minimum or below (<Dfl 1400 for 1 person, <Dfl 1900 for >=2)/higher. Occupational status was measured using present or last occupation of people classified into five groups (un/semiskilled manual; skilled manual; farmer/self-employed; routine non-manual; professionals/ managers);34 it was dichotomized as: semi- or unskilled work/ higher. Educational level referred to the highest level reached and was classified into four groups (<=basic education/junior secondary or vocational/senior secondary or vocational/ vocational colleges or university);33 it was dichotomized as: only primary education/higher.

Analysis
Socioeconomic differences in the validity of self-reported use of health care were estimated by comparing SE groups regarding utilization rates on the basis of self-report and of registration, and regarding concordance of self-report and registration. The latter was measured by (Cohen's) kappa (agreement adjusted for chance agreement)35,36 and the proportions of actual agreement.37 In these comparisons, the lowest SE group was compared with all others.

Next, all the analyses were repeated using hierarchical log-linear analyses, because of the sensitivity of kappa for differences in prevalence.38–40 All log-linear models included reported and registered use of health care, and SES. Socioeconomic differences in the concordance of reported and registered use of health care were measured by the second order interaction term between these variables and SES, in models already containing all main effects and first order interactions.38,39,41 Subsequently, the most parsimonious models were selected in an exploratory backward elimination process (with P-out set at 0.05). These analyses were performed for the dichotomized indicators of SES and for the more detailed categorizations.

Third, we computed kappas between odds ratios (OR) for reported and registered utilization including all categories of the combinations of SE indicators and types of care for which the previous analysis yielded statistically significant results.

Finally, we computed OR for utilization by each (detailed) SES measure, on the basis of both self-report and registration. All analyses were performed with the SPSS 10.0 for Windows statistical package.42

Results

The differences between utilization rates on the basis of self-report and of registration were similar for all SE groups studied, although self-report was generally higher (Table 1Go). The overall concordance between reported and registered use of care was good to excellent35 for all three types of care: kappa varied from 0.60 to 0.80. Regarding hospitalizations and physiotherapy, kappas were very similar across SE groups. Regarding prescription drugs, however, they were somewhat lower for respondents of low educational and occupational status, but without statistical significance.

Hierarchical log-linear analyses regarding the dichotomized indicators of SES gave results broadly similar to those for kappa. Inclusion of more detailed categorizations of SES in these models yielded similar results except in one case. Regarding hospitalization, the second order interaction with occupational status in five levels contributed to the model (P = 0.025). Regarding this combination, kappa was relatively low for the ‘skilled manual workers’ category and the ratio of reported versus registered utilization also differed from all the others. Furthermore, occupational status contributed to the prediction of hospitalization in the crude model for reported use whereas it did not for registered use (Table 2Go).


View this table:
[in this window]
[in a new window]
 
Table 2 Concordance between registration and self-report estimated by Cohen's kappa, and odds ratios (OR, 95% CI) for reported and registered utilization, regarding hospitalization by occupational level
 
The OR for utilization by each (detailed) SES measure, also differed relatively little between self-report and registration and there was no systematic trend regarding differences in OR across SE categories. As an example of these differences, we therefore only present OR comparing the lowest with the highest level and the ratio of the OR based on self-report and registration, and overall P-values (Table 3Go). In two instances, SES contributed to the prediction of registered utilization, but in no case of reported utilization. Comparisons of the lowest SE category with all others yielded similar results. However, differences between OR regarding self-report and registration were smaller (range of ratios: 0.77–1.22) and SES did not contribute to the prediction of utilization.


View this table:
[in this window]
[in a new window]
 
Table 3 Odds ratios (OR, 95% CI) for utilization of the lowest socioeconomic (SE) category compared to the highest (detailed) one, after adjustment for differences in age and gender
 
Discussion and Conclusion

The results of our study show that the concordance between reported and registered utilization of health care is generally good to excellent and differs relatively little between people of low SES and others (i.e. the lower and middle income groups). If SE groups are assessed in more detail, differences in concordance are still mostly without statistical significance and systematic trends, though variation is somewhat larger.

Our results indicate that self-report offers a reasonably valid estimate of SE differences in utilization of health care, though it may add some measurement error if one compares it with registration data and may yield somewhat different risk estimates. Concordance between self-report and registration is rather similar across SE categories and does not vary more than could be expected on the basis of chance. Perfect agreement can seldom be expected because of real differences between the phenomena that are reported and registered. For instance, the provision of a prescribed drug does not imply its immediate use and the reverse also applies. Regarding physiotherapy, registration of payment for a treatment does not always correspond with actual use. This probably also contributes to the generally higher utilization of care on the basis of self-report that we found. However, imperfect recall probably lowers concordance too, especially if it concerns a less intrusive type of care. The relatively low kappas for prescription drugs offer some indication for this, as such use of drugs has almost certainly less impact than a hospitalization. Other, spurious, causes of a lower concordance, such as erroneous linkage and incomplete registration are unlikely.30

The results of most other studies fit with our results regarding the fairly high concordance between self-reported and registered use of medical and of hospital care among the general population.20–23 Regarding hospitalization in New Zealand, Norrish et al. found no differences in recall of admissions during a 4-year period by social class.20 In the US, Roberts et al. generally found high agreement between self-report and registration over a one-year period without differences by SES.21 Also in the US, Weissman et al. did not find differences by SES either.22 Finally, Bergmann et al. assessed the validity of self-reported diagnoses associated with hospitalization, again in the US.23 They report better validity among those with a high educational level but do not present data regarding overall utilization rates.

Contrary to our health survey, the majority of these studies were prospective or concerned at least predefined cohorts.20–24 The validity of self-report may in general be expected to be better in this case, as people have volunteered to participate in a more demanding design. The reverse of this is that the response rate in our study was only 61.4%. This may have influenced our results, for instance because the least motivated people did not respond at all but would have been discordant more often if they had been respondent. However, a formal analysis of the non-response in this study does not show major differences in utilization and in the association of utilization with background characteristics between respondents and non-respondents.29

Only Cleary and Jette,25 and Bellon and coworkers18 have compared registered data on health care utilization with self-report from cross-sectional studies among the general population. Cleary and Jette found that in the US the reporting errors regarding use of outpatient medical care in the preceding year were inversely associated with income but not with educational level and occupational status, in a study of 908 people. After adjustment for age, the association with income disappeared.25 Bellon and coworkers assessed the validity of self-reported utilization of primary care over a range of reporting periods (2 weeks–1 year) among 656 people in Spain. They found that this was not associated with employment status, although both the under- and overreporting was somewhat higher for those without schooling (though in only one out of the eight combinations that they assessed with statistical significance).18 Bellon et al.,18 as well as Glandon et al.,24 and Cleary and Jette,25 found some tendency for overreporting among people with a poor health status. This might have contributed to a lower concordance among those with low SES (as these people more often have a poor health status), and offer an explanation for this, but its overall effect appeared to be limited. Evidence on the other types of care studied, such as physiotherapy and further use of prescription drugs, is completely lacking.

Our study covered native Dutch with HIF insurance only, which may have influenced our results in two ways. Firstly, this may explain the relatively small SE differences in utilization, smaller than those found in previous Dutch studies, because of three mechanisms.11,43 As mentioned earlier, HIF insurance covers lower and middle income groups, i.e. the lowest 63% of incomes in the Netherlands. Thus, the variation in income in our study is about half of that in the total Dutch population: in 1998, the mean yearly net incomes of households in the 1st (the lowest), 6th, and 10th (the highest) decile of the Dutch income distribution were about Dfl 9000, 49 000, and 112 000, respectively.44 If utilization decreases as income increases, this will lead to smaller SE differences in our study than in previous ones. Next, type of insurance is associated both with income and with utilization.43,45,46 It partially mediates the relation of the latter in the Netherlands, in such a way that SE differences are largest if type of insurance is not adjusted for.43,45,46 Finally, non-native Dutch mostly have a low SES and use some types of care relatively frequently.27,31 Because of these factors, SE differences in utilization may be expected to be smaller in this study than in previous ones.43,45,46

The second question, because of the restriction of our study to native Dutch with HIF insurance, is whether our results can be generalized to other studies on SE differences in utilization of care. With regard to the range of incomes that are included, our study shows no systematic differences in concordance by increasing SES, over quite a broad range of incomes. This makes it rather unlikely that concordance will be better for higher income groups, but we cannot rule this out. Regarding country of birth, the results as presented are limited to native Dutch. Previous analyses have shown a somewhat lower concordance among some non-native Dutch in our original sample, especially among Turks and Moroccans.30 If these had been included in the present analysis study, concordance would have been lower among the low SES group, because of the lower mean SES of this group. Regarding the generalizability of our results, however, the proportion of immigrants is much higher in the population of Amsterdam than in the general population of any large country in the European Union or the US (for instance, in 1995, 27% of the population of Amsterdam was born outside the Netherlands, and 9% of the total Dutch population).27 Therefore, results covering only the native Dutch can be generalized better to these countries than if non-natives had been included, even though this leads to the exclusion of a part of our original sample, however, it covers only the lower and middle income groups.

In conclusion, self-reports in population health surveys mostly appear to give valid estimates of SE differences in utilization of health care. This conclusion is corroborated by the findings from the single community-based study that assessed differences across SE groups in utilization, based on registered utilization and self-reported SES.14 This study found very similar results to a previous, also Canadian, community-based study that was based on self-reported utilization and SES.9 Its results also fit into the picture of previous studies on specific groups and types of care.18,20–25 Future studies are needed to confirm the validity across SE groups of self-reported utilization of various types of care in general population samples from other countries, and across the entire income range.

Appendix

Questions included in the interview

Have you been admitted to a hospital during the preceding year (i.e. the past 12 months)? [no/yes] If yes, how often? [number of times]

Have you used the following services in the past year (i.e. the past 12 months)? A physiotherapist or masseur (not counting hospital admissions). [no/yes] If yes, how often? [number of times]

Have you taken any medication prescribed by a doctor (other than the ‘pill’) in the past three months? [no/yes]


KEY MESSAGES

  • Socioeconomic differences in self-reported use of health care are well known.
  • Evidence on the validity of self-reported utilization of health care across socioeconomic groups is limited.
  • This study shows that concordance between reported and registered utilization is generally good to excellent, for native Dutch with low or middle incomes.
  • For these people, concordance differed little between those with a low socioeconomic status and others.
  • Self-report offers a reasonably valid estimate of differences in utilization of health care between socioeconomic groups, at least for lower and middle income groups.

 

Acknowledgments

The Amsterdam Municipal Health Service (GG&GD) provided the data for this study. The fieldwork for these data was financially supported by the Municipality of Amsterdam (GG&GD), eight submunicipal authorities, the Amsterdam Municipal Health Service, and the Amsterdam Social Health Insurance Company (ZAO). The analyses were financially supported by the Social Health Insurance Company (ZAO), TNO Prevention and Health, and the Amsterdam Academic Medical Center.

References

1 Kunst AE, Mackenbach JP. The size of mortality differences associated with educational level in nine industrialized countries. Am J Public Health1994;84:932–37.[Abstract]

2 Pappas G, Queen S, Hadden W, Fisher G. The increasing disparity in mortality between socioeconomic groups in the United States, 1960 and 1986. N Engl J Med1993;329:103–09.[Abstract/Free Full Text]

3 Sorlie PD, Backlund E, Keller JB. US mortality by economic, demographic, and social characteristics: the National Longitudinal Mortality Study. Am J Public Health1995;85:949–56.[Abstract]

4 Smith GD, Neaton JD, Wentworth D, Stamler R, Stamler J. Socioeconomic differentials in mortality risk among men screened for the Multiple Risk Factor Intervention Trial: I. White men. Am J Public Health1996;86:486–96.[Abstract]

5 Smith GD, Wentworth D, Neaton JD, Stamler R, Stamler J. Socioeconomic differentials in mortality risk among men screened for the Multiple Risk Factor Intervention Trial: II. Black men. Am J Public Health1996;86:497–504.[Abstract]

6 Mackenbach JP, Kunst AE, Cavelaars AEJM, Groenhof F, Geurts JJM. Socioeconomic inequalities in morbidity and mortality in western Europe. Lancet1997;349:1655–59.[ISI][Medline]

7 Mackenbach JP, Kunst AE, Groenhoff F et al. Socioeconomic inequalities in mortality among women and among men: an international study. Am J Public Health1999;89:1800–06.[Abstract]

8 Ben-Shlomo Y, White I, McKeigue PM. Prediction of general practice workload from census based social deprivation scores. J Epidemiol Community Health1992;46:532–36.[Abstract]

9 Katz SJ, Hofer TP, Manning WG. Physician use in Ontario and the United States: the impact of socioeconomic status and health status. Am J Public Health1996;86:520–24.[Abstract]

10 Carr-Hill RA, Rice N, Roland M. Socioeconomic determinants of rates of consultation in general practice based on fourth national morbidity survey of general practices. Br Med J1996;312:1008–13.[Abstract/Free Full Text]

11 Van der Meer JB, van den Bos J, Mackenbach JP. Socioeconomic differences in the utilization of health services in a Dutch population: the contribution of health status. Health Policy1996;37:1–18.[ISI][Medline]

12 Alberts JF, Sanderman R, Eimers JM, van den Heuvel WJ. Socioeconomic inequity in health care: a study of services utilization in Curacao. Soc Sci Med1997;45:213–20.[ISI][Medline]

13 McIsaac W, Goel V, Naylor D. Socio-economic status and visits to physicians by adults in Ontario, Canada. J Health Serv Res Policy1997; 2:94–102.[Medline]

14 Kephart G, Thomas VS, MacLean DR. Socioeconomic differences in the use of physician services in Nova Scotia. Am J Public Health1998; 88:800–03.[Abstract]

15 Van der Meer JBW. Equal Care, Equal Cure: Socieconomic Differences in the Use of Health Services and the Course of Health Problems (PhD thesis). Rotterdam: Erasmus Univerisity, 1998.

16 McNiece R, Majeed A. Socioeconomic differences in general practice consultation rates in patients aged 65 and over: prospective cohort study. Br Med J1999;319:26–28.[Abstract/Free Full Text]

17 Reid FD, Cook DG, Majeed A. Explaining variation in hospital admission rates between general practices: cross sectional study. Br Med J1999; 319:98–103.[Abstract/Free Full Text]

18 Bellon JA, Lardelli P, Luna JD, Delgado A. Validity of self reported utilisation of primary health care services in an urban population in Spain. J Epidemiol Community Health2000;54:544–51.[Abstract/Free Full Text]

19 Dunlop S, Coyte PC, McIsaac W. Socio-economic status and the utilisation of physicians' services: results from the Canadian National Population Health Survey. Soc Sci Med2000;51:123–33.[ISI][Medline]

20 Norrish A, North D, Kirkman P, Jackson R. Validity of self-reported hospital admission in a prospective study. Am J Epidemiol1994;140:938–42.[Abstract]

21 Roberts RO, Bergstrahl EJ, Schmidt L, Jacobsen SJ. Comparison of self-reported and medical record health care utilization measures. J Clin Epidemiol1996;49:989–95.[ISI][Medline]

22 Weissman JS, Levin K, Chasan-Taber S, Massagli MP, Seage GR 3rd, Scampini L. The validity of self-reported health-care utilization by AIDS patients. AIDS1996;10:775–83.[ISI][Medline]

23 Bergmann MM, Byers T, Freedman DS, Mokdad A. Validity of self-reported diagnoses leading to a hospitalization: a comparison of self-reports with hospital records in a prospective study of American adults. Am J Epidemiol1998;147:969–77.[Abstract]

24 Glandon GL, Counte MA, Tancredi D. An analysis of physician utilization by elderly persons: systematic differences between self-report and archival information. J Gerontol1992;47:S245–52.[ISI][Medline]

25 Cleary PD, Jette AM. The validity of self-reported physician utilization measures. Med Care1984;22:796–803.[ISI][Medline]

26 Reijneveld SA. The impact of individual and area characteristics on urban socioeconomic differences in health and smoking. Int J Epidemiol1998;27:33–40.[Abstract]

27 Reijneveld SA. Reported health, lifestyles and use of health care of first-generation immigrants in the Netherlands: do socioeconomic factors explain their adverse position? J Epidemiol Community Health1998;52:298–304.[Abstract]

28 Reijneveld SA, Verheij RA, De Bakker DH. The impact of area deprivation on differences in health: does the choice of the geographical classification matter? J Epidemiol Community Health2000;54:306–13.[Abstract/Free Full Text]

29 Reijneveld SA, Stronks K. The impact of response bias on estimates of health care utilisation in a metropolitan area: a study using administrative data. Int J Epidemiol1999;28:1134–40.[Abstract]

30 Reijneveld SA. The cross-cultural validity of reported use of health care: a comparison of survey and registration data. J Clin Epidemiol2000;53:267–72.[ISI][Medline]

31 Stronks K, Ravelli ACJ, Reijneveld SA. Immigrants in the Netherlands: equal access for equal needs? J Epidemiol Community Health2001; 55:701–07.[Abstract/Free Full Text]

32 Liberatos P, Link BG, Kelsey JL. The measurement of social class in epidemiology. Epidemiol Rev1988;10:87–121.[ISI][Medline]

33 De Bruin A, Picavet HSJ, Nossikov A (eds). Health Interview Surveys; Towards International Harmonization of Methods and Instruments. Copenhagen: WHO, 1996 (WHO reg. publications; European series, no. 58).

34 Erikson R, Goldthorpe JH, Portocarero L. Intergenerational class mobility in three western countries: England, France and Sweden. Br J Sociol1979;30:415–41.[ISI]

35 Armitage P, Berry G. Statistical Methods in Medical Research. 3rd Edn. Oxford: Blackwell, 1995.

36 Garner JB. The standard error of Cohen's kappa. Stat Med1991;10:767–75.[ISI][Medline]

37 Cicchetti DV, Feinstein AR. High agreement but low kappa: II resolving the paradoxes. J Clin Epidemiol1990;43:551–58.[ISI][Medline]

38 Fienberg SE. The Analysis of Cross-classified Categorical Data (5th Print). Cambridge: MIT Press, 1987.

39 Agresti A. Categorical Data Analysis. New York: John Wiley, 1990.

40 Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol1993;46:423–29.[ISI][Medline]

41 Becker MP, Agresti A. Log-linear modelling of pairwise interobserver agreement on a categorical scale. Stat Med1992;11:101–14.[ISI][Medline]

42 SPSS for Windows, Version 10.0.5. Chicago: SPSS, 1999.

43 Statistics Netherlands. Netherlands Health Interview Survey 1981–1995. The Hague: Sdu Publishers, 1996.

44 Statistics Netherlands. Number and average income of households per 10%-group of disposable income. http://www.cbs.nl/en/figures/ keyfigures/siv1556w.htm (Last update: 25–01–2000).

45 Reijneveld SA. Socioeconomic Differences in Health of the Amsterdam Population (in Dutch). Amsterdam: Municipal Health Service, Dept. of Epidemiology, 1997.

46 Bongers IM, van der Meer JB, van den Bos J, Mackenbach JP. Socio-economic differences in general practitioner and outpatient specialist care in The Netherlands: a matter of health insurance? Soc Sci Med1997;44:1161–68.[ISI][Medline]