May telephone surveys provide reliable public health surveillance data for municipalities? Mode effects differ between categories of questions. The HUNT Study, Norway

Abstract

Background: Availability of data on health and its determinants at the local area level is a prerequisite for developing interventions and public health campaigns locally. Collecting self-reported data by means of telephone interviews may rapidly provide relevant data. The reliability of such data may be questioned. In this study, we sought to compare exact similar questions addressed by a recent telephone survey with a previous large scale and very comprehensive population health survey (The Nord-Trøndelag Health Study 2006-08 – HUNT3), conducted a few years earlier in the same geographical region. This was done in order to examine the reliability of telephone interviews as a method to provide data on health and determinants to enable municipal authorities to get a sufficient overview.
Methods: One rural and one urban municipality covered by HUNT3 using paper questionnaires were resurveyed through computer assisted telephone interviews. The weighted results for 34 dichotomized variables were compared using chi square tests.
Results: The comparison of results between the rural and the urban samples and HUNT3 involved 68 chi square tests, 25 of which (38%) displayed significant differences. The ability of the telephone survey to replicate the results from HUNT3 was only moderate, but with differences between survey themes. Comparability was poor for adverse life events and mental health factors, fair for behavioural and risk factors, and skewed for general health and life satisfaction. The replication was good for reports on the less sensitive and subjective theme of cultural participation.
Conclusion: The comparability of the data differed between themes. The differences may be ascribed to mode effects and to some extent the time lag between the surveys. Because replicability on issues that may be more embarrassing or stressful to recall appears to be poorer, and the more subjective self-assessments of health and well-being appear skewed, it is reasonable to conclude that there is an interviewer effect in the telephone survey. The use of a questionnaire through mail or web to monitor public health in municipalities should be considered as an alternative.