A coda: oversimplification, implicit assumptions, and measurement error

Mark S Gilthorpe1, Yu-Kang Tu1 and David Gunnell2

1 Biostatistics Unit, Centre for Epidemiology and Biostatistics, Leeds Institute of Genetics, Health & Therapeutics, University of Leeds, Leeds LS2 9LN, UK
2 Department of Social Medicine, University of Bristol, Bristol BS8 2PR, UK. E-mail: d.j.gunnell{at}bristol.ac.uk

This debate demonstrates the range of approaches, and their associated limitations, used to examine this deceptively complex issue. An understanding of the range of relationships of errors with each other and with the unobserved measures is crucial. We concur with all contributors that our pieces are an oversimplification, making various implicit assumptions that were omitted in the interests of simplicity and brevity; it is valuable to receive further comments and insights from other correspondents. We summarize what we feel are the main conclusions to draw from this debate.

The debate on underlying assumptions is probably more philosophical than statistical. There are differences in the adopted definitions of ‘true’ outcome and associated assessment errors. For instance, Ian White1 begins with the assumption that over-/under-reporting is present and therefore assessment errors are correlated with unobserved values for self-report whilst uncorrelated for measurement. Consequently, using Oldham's method to test for unequal variances is only valid if error variances of self-report and measurement are equal, which one cannot readily assess. In contrast, we made no assumptions about the correlation between assessment errors and unobserved values, as this is what we sought to test, though we assumed equal error variances. Oldham's method is then valid. The paradox of a method being valid/ invalid with different assumptions for the same problem further reflects the complexity of this issue!

Multilevel modelling requires various assumptions concerning the error structures to be explicit, and this indeed is no bad thing. One assumption of the multilevel approach not made explicit was that the ‘measure-type’ variance at level-2 and the intercept variance at level-1 could not be estimated simultaneously, as the model would be over-identified. This was circumnavigated by setting the level-1 variance to zero, though the ‘measure-type’ covariate must be centred to avoid overestimation of the level-2 covariance. If we assume unequal error variances (for which there is no obvious means of verification), the bivariate approach of Rasbash and Goldstein2 would be more appropriate.

Within Oldham's method extended to multiple regression, one may seek to identify if over-/ under-reporting is associated with a covariate. If an association is present, mathematical coupling (MC) is re-introduced, as agreed by all correspondents. However, a potentially much greater bias occurs if the covariate is also associated with the mean outcome, due to what is recognised in the literature as Lord's paradox.3 This is illustrated in the Figure 1 for the covariate gender, where we assume that males generally report/measure taller than females and that tall people under-report. Bias occurs for continuous covariates also, though this phenomenon is then better known as the reversal paradox.4 As suggested, regressing self-report on measurement would provide an alternative approach, providing appropriate adjustment was made for regression to the mean due to assessment error (usually unknown and only estimated).5



View larger version (15K):
[in this window]
[in a new window]
 
Figure 1 Difference between self-reported (X1) and measured (X0) outcomes

 
Finally, with specific regard to MC, the discussion surrounding measurement error and associated assumptions should not detract from the key issue that when MC is present the standard null hypotheses are no longer valid.6,7 This is why the methods in the original article are not acceptable.8 MC alone may not be a good guide to the presence (or absence) of many potential model difficulties, but its presence ALWAYS predicates distortion of the null hypotheses.


    References
 Top
 References
 
1 White IR. Assessing correlation between reporting errors and true values: untestable assumptions are unavoidable. Int J Epidemiol 2004; 33:1400–01.[Free Full Text]

2 Rasbash J, Goldstein H. Mathematical coupling: a simpler approach. Int J Epidemiol 2004; 33:1401–02.[Free Full Text]

3 Lord FM. A paradox in the interpretation of group comparisons. Psychol Bull 1967; 68:304–05.[ISI][Medline]

4 Stigler SM. Statistics on the Table. Cambridge, Massachusetts: Harvard University Press; 1999.

5 Blomqvist N. On the relation between change and initial value. J Am Stat Assoc 1977; 72:746–49.[ISI]

6 Gunnell D, Berney L, Holland P et al. Does the mispreporting of adult body size depend upon an individual's height and weight? Methodological debate. Int J Epidemiol 2004; 33:1398–99.[Free Full Text]

7 Gilthorpe MS, Tu Y-K. Mathematical coupling: a multilevel approach. Int J Epidemiol 2004; 33:1399–400.[Free Full Text]

8 Gunnell DJ, Berney L, Holland P et al. How accurately are height, weight and leg length reported by the elderly and how closely are they related to measurements recorded in childhood. Int J Epidemiol 2000; 29:456–64.[Abstract/Free Full Text]





This Article
Extract
Full Text (PDF)
All Versions of this Article:
33/6/1402    most recent
dyh305v1
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Similar articles in PubMed
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Request Permissions
Google Scholar
Articles by Gilthorpe, M. S
Articles by Gunnell, D.
PubMed
PubMed Citation
Articles by Gilthorpe, M. S
Articles by Gunnell, D.