Published: June 5, 2018

Download: Life Course Centre Working Paper: 2017-02


Francisco Perales and Bernard Baffour.

An updated version of this paper has been published in Survey Research Methods. DOI: 10.18148/srm/2018.v12i2.7225

Non-Technical Summary:

Mental health conditions are amongst the largest causes of disease burden at a global level, and understanding the predictors and consequences of ill mental health is a fundamental goal of health research, policy and practice. Many studies of mental health rely on the analysis of population surveys. However, this research makes one important assumption, namely that the accuracy of the information gathered in surveys is comparable for individuals with low and high levels of mental health. This is problematic, as there are reasons to expect poorer survey interview outcomes amongst individuals with ill mental health, which may in turn lead to less accurate responses to survey questions.

In this study, we fill a gap in knowledge by comparing interviewer ratings of the quality of the survey interview (IRQSI) between respondents with poorer and better mental health. We consider three aspects of IRQSI: (i) interviewer ratings of survey respondents being suspicious of the study, (ii) having issues understanding the survey questions, and (iii) being uncooperative. Survey methodology manuals emphasize the importance of respondent trust, cooperation and understanding in the survey interview situation, as poor performance in these dimensions may affect survey estimates by leading to higher missing data, measurement error and report bias.

Our findings are consistent with expectations: individuals with poorer mental health are more likely to display low IRQSI. These associations were visible across a range of IRQSI outcomes and measures of mental health and disorders. These observed deficits in IRQSI amongst respondents with poor mental health constitute new and important knowledge, with implications for how researchers undertake survey research on mental health and how they interpret the results. To the extent that professionally-trained interviewers are accurate in their assessments, this finding is suggestive that the accuracy of the resulting survey data is comparatively lower amongst respondents with poor mental health. Hence, it is possible that survey analyses of individuals with poor mental health produce unreliable results, which poses a challenge to the usefulness of findings generated using survey data to inform the design of evidence-based mental health policy.

We conclude that, while surveys are powerful means by which to gather evidence to inform the development of health policies, it is not clear that researchers and policymakers should take the accuracy of survey data generated from respondents with ill mental health for granted. More research aimed at comparing how individuals with poorer and better mental health engage in the survey process, and whether and how their poor mental health is related to the quality of the information retrieved from these individuals is sorely needed.


January 6, 2017