Paper summary: The benefits of random sampling

Lessons from the 2015 UK General election

This paper presents data from the 2015 British Social Attitudes survey on reported voting behaviour in the 2015 General Election. It is a pre-release from our 33rd report, published in June 2016. It explores why these data are more accurate than the opinion polls that took place before the election in replicating what actually happened at the election - both in terms of voter turnout and levels of support for the main political parties.

Key findings

• Data from the British Social Attitudes survey suggest that unrepresentative samples caused the General Election polling error.
• Data on reported voting from BSA 2015 show a Conservative lead over Labour of 6.1 points, very close to the actual lead recorded at the 2015 General Election of 6.6 points.
• 70% of BSA respondents reported voting at the General Election, close to the official turnout figure of 66%.

Introduction

At the 2015 general election, the opinion polls collectively underestimated the Conservative lead over Labour. In their final polls they suggested on average that the two parties were neck and neck on 34% each, when in the event the Conservatives proved to be seven points ahead. Moreover, even when the polling companies reinterviewed those whom they had polled before the election, they still obtained much the same neck and neck result.

This paper reports the results that were obtained by NatCen’s 2015 British Social Attitudes survey (BSA) when it asked all of its 4,328 respondents whether and how they voted in the election. BSA is conducted very differently from the opinion polls. Whereas the polls interview those who can be contacted during a short period of time on the phone or over the internet, BSA is conducted face to face over a period of months amongst a randomly selected sample of respondents, with interviewers making multiple attempts to secure interviews with those selected for interview. The paper argues why this means the polls are at greater risk of obtaining samples of respondents who are unrepresentative of the nation at large.

In contrast to the polls, BSA 2015 replicates the outcome of the election quite closely. Just 70% said that they had voted, only a little above the official turnout figure of 66%. Meanwhile, amongst those who did say that they voted, support for the Conservatives is six points higher than that for Labour, very close to the near seven point lead actually recorded in the election.

Much of the speculation about the failure of the polls in the general election has focused on claims that voters were not being honest with the pollsters – and perhaps even to themselves. However, BSA’s relative success in replicating the outcome of the 2015 election suggests that the problem of the polls lay instead in the unrepresentative character of the samples of people whom they interviewed, an error they were then unable to correct in their subsequent analysis and weighting of their data. The paper reports the results of some analyses of BSA’s data that suggest where the source of that error may lie, looking in particular at the importance of identifying accurately those who are less likely to vote and the impact of an apparent tendency for Labour voters to be more accessible to pollsters.

Download the full paper