Differential turnout between polls and the election
Differential turnout between polls and the election
Our final pre-election survey was sent out to just under 3,000 people on Monday 4th May and was completed by the end of Tuesday 5th May before being published on Wednesday the 6th.
In that poll 81% said that they were likely to vote and told us which party they would vote for compared to an official turnout of 66%. Even restricting to those who ?would definitely vote? gives us 76%, still significantly above the actual turnout.
In fact if we look at other polls we see similar results.
- ICM?s final voting intention figures come from 1,544 respondents out of 2,023 equivalent to 76%
- In YouGov?s final poll of 10,307 people, 76% said that they were 10/10 certain to vote although it is unclear how many of these are included in their final voting intention figures
- For Ipsos MORI 873 out of 1,096 respondents (80%) say that they were absolutely certain to vote
In all cases further adjustments are made to the final figures but the point of bringing these numbers up is that people who respond to surveys, whether on the telephone or online, are more likely to say that they will vote than actual turnout implies. This is either an example of social desirability bias ? saying that you will do something even if you won?t because it?s the sort of thing that you think you should do ? or evidence that people who respond to polls generally are more politically engaged and enthusiastic than the population at large.
Either we are counting non-voters who themselves do not turnout on the day or we are counting too many people who are voting from groups that do not turnout in the numbers suggested by our polls.
In either case, polls exaggerate turnout but because different demographic groups turnout at different rates, this has a disproportionate influence on some groups and thus on final vote share.
So what is the effect of this?
The table below shows the different effects on each age group using Ipsos MORI?s ?how Britain voted? turnout figures. Obviously those figures themselves being from surveys are potentially subject to the same error but, with that caveat in mind, what they tell us is interesting.
| Age group | Implied turnout in our poll | 2015 turnout in age group (source) | Implied turnout as a % of 2015 turnout | % voting Labour in poll | % voting Conservative in poll | % voting Green in poll |
| ALL | 81% | 64% | 123% | 34% | 35% | 6% |
| 18-24 | 64% | 43% | 149% | 41% | 32% | 11% |
| 25-34 | 78% | 54% | 144% | 39% | 32% | 8% |
| 35-44 | 77% | 64% | 120% | 38% | 29% | 8% |
| 45-54 | 86% | 72% | 119% | 36% | 22% | 6% |
| 55-64 | 84% | 77% | 109% | 32% | 33% | 7% |
| 65-74 | 88% | 78%* | 113% | 26% | 45% | 4% |
| 75+ | 92% | 78%* | 123% | 27% | 44% | 0% |
*combined 65+ figure
As we can see, implied turnout is higher for all age groups but particularly so for those under age 35 who are also the most pro-Labour. This means that these groups formed a larger share of our poll?s ?voters? than they should. This group is also the most pro-Green although the Greens are overstated by all groups relative to their actual share so this only addresses part of the problem there.
Let?s look also at socio-economic grade. This is trickier as the classification can vary depending on who is doing the classifying but, even with this caveat in mind, it still looks like our poll is over-representing the more pro-Labour groups of the population.
| Socio-economic group | Implied turnout in our poll | 2015 turnout in age group (source) | Implied turnout as a % of 2015 turnout | % voting Labour in poll | % voting Conservative in poll | % voting Green in poll |
| ALL | 81% | 64% | 123% | 34% | 35% | 6% |
| AB | 81% | 65% | 125% | 34% | 35% | 6% |
| C1 | 81% | 66% | 122% | 35% | 37% | 6% |
| C2 | 80% | 58% | 137% | 35% | 31% | 8% |
| DE | 78% | 57% | 136% | 41% | 29% | 5% |
The figures for vote share and likely turnout are all determined by asking them of respondents directly before weighting such as party propensity kicks in. Whether this disparity is due to social desirability bias towards voting, or the fact that polls generally are answered by a more politically engaged section of the population, the fact remains that direct questions alone are clearly not enough to accurately isolate the voting population.
Looking again at our final poll
Taking the biases exposed above we have adjusted the numbers in our final poll to see what effect correcting for them would have had.
Going back to how we originally put it together, our voting intention polls go to a selection of respondents on our consumer panel designed to be nationally representative according to a number of demographic factors. The sample that we ultimately achieve tends to be very close to these targets but we then use weighting to make the last few adjustments to match them.
We then apply party propensity weighting. The full explanation of this is here but in essence it makes sure that our sample is representative politically as well as demographically by asking voters how likely they are to ever vote for each party on a scale from 1-10 and from that we put them into categories such as ?Labour ? lean right? or ?Conservative ? lean left?. We know how large or small each of these groups should be and so we can weight our sample to ensure the correct balance.
In the table below you can see the original raw figures and the effect that each stage of weighting and adjustment had on them in our final poll:
| Final Opinium pre-election poll | Raw figures | With demographic weighting | With demographic and party propensity weighting |
| Conservative | 34% | 33% | 35% |
| Labour | 33% | 33% | 34% |
| UKIP | 13% | 14% | 12% |
| Liberal Democrats | 8% | 8% | 8% |
| SNP | 5% | 5% | 4% |
| Green | 5% | 5% | 6% |
The next table shows what happens if we correct for some of the biases mentioned earlier and we have made each change in stages to be as clear as possible.
We start again with the original unweighted figures and then add demographic weighting with the 7-way age split (18-24, 25-34, 34-44, 45-54, 55-64, 65-74, 75+). This corrects for the under-representation of those aged 65+. We then add turnout corrections to make our ?voting population? match the real one and remove the over-representation of groups like 18-24 year olds and DE voters. Finally we add our party propensity weighting.
| Final Opinium pre-election poll | Raw figures | Demographic weighting with 7-way age split | Demographic weighting with 7-way age split AND turnout corrections | Demographic weighting with 7-way age split AND turnout corrections AND party propensity weighting |
| Conservative | 34% | 33% | 34% | 37% |
| Labour | 33% | 33% | 32% | 32% |
| UKIP | 13% | 14% | 15% | 12% |
| Liberal Democrats | 8% | 8% | 8% | 8% |
| SNP | 5% | 4% | 4% | 4% |
| Green | 5% | 5% | 5% | 6% |
Conclusions
It is the easiest thing in the world after an election to take your final prediction, tweak it here and there until it looks like the actual result and then claim that this is how to do it in future. That is manifestly not what we are doing here.
The adjustment that has had the most significant effect has been to include the turnout corrections, themselves based on the assumption that the 2015 electorate would have similar patterns of differential turnout to the 2010 electorate. Applying these to future elections is potentially problematic.
US pollsters construct intricate likely-voter models using similar approaches to the above and which use demographic information and past elections to predict likely turnout. But this can be problematic if the composition of that voting population changes as a great deal of weight is then put on what predictions a polling company might make about that composition. Given the perceived closeness of the 2015 British election most expected turnout to be higher than in 2010, perhaps as high as 70%. In fact turnout barely increased by one percent. This means that applying a turnout filter to match the 2010 voting population would have been more accurate but we only know this after the fact.
