Avoid psephological apophenia!

Posted by on March 14, 2014 in Articles, Progress | 0 comments

Avoid psephological apophenia!

This post originally appeared on Progress Online.

One of the problems of writing about opinion polls is that the more startling, extraordinary and news-worthy a poll finding is, the more likely it is to be wrong. The fact that most of them are commissioned by the media accentuates the tendency to highlight the sensational results, particularly the ones your own media organisation has commissioned. Twitter may have amplified this trend, with partisans eagerly tweeting results that look good for their party and downplaying the ones that bear bad news. Variations in headline numbers usually reflect the inevitable workings of the science of sampling rather than real movements in public opinion; it is as well to wait a while and look at more than one poll to see if there is a real trend. Good writers on polls such as Anthony Wells of UK Polling Report and the Nottingham team who run Polling Observatory often find themselves writing variations of ‘nothing much to see here, Labour still a bit ahead’. I’ll try to go a little beyond that here, but first a word on the national polls.

There might still be nothing much to see in the national polls at the moment; Labour remains ahead by a fairly small but extremely stubborn margin. There have been several polls in the last few days suggesting that Labour’s lead is at the low end of its recent range, with IPSOS-MORI, ICM, Populus and YouGov all finding Labour leads in the 1-3 point zone, but last weekend there were some with rather large Labour leads of 7 and 9 points. We can’t tell yet whether Labour’s lead has really slipped from 6-ish to 4-ish, but it certainly can’t be ruled out either and there is increasing evidence that Labour might have suffered a further erosion in support from around 38 per cent to around 36 per cent. It is quite possible that if Labour’s lead really is around 3 points, a good poll well within the standard sampling variation could put the Conservatives a point or two ahead. The chances of at least one doing so are heightened by the frequency with which polls are conducted.

There do not seem to have been any specific events that might have triggered a slippage, but it is possible that it reflects a slowly increasing level of personal economic optimism – the proportion YouGov have found expecting better times has edged up to around 17 per cent (although it is still less than most polls in the run-up to the 2010 election). Conservatives, as one might expect, are most optimistic but interestingly UKIP supporters are the most pessimistic. One can have all sorts of fun with time series and fitting trend lines (and Stephen Tall does so here) but I am wary of drawing too many conclusions yet.

When opinion polls are published, polling companies – quite rightly – publish all sorts of detail beyond the headline figure of who is ahead. They give a lot of information about how strongly respondents are committed to voting and how different groups of voters are thinking about the parties and the issues. The basic breakdown is by sex, age, social class, region or nation and usually by their recalled vote in the 2010 general election.

Even though it must be better than secrecy (and parties’ bogus claims about what their private polls supposedly show), transparency is not without its problems. There are two big temptations when it comes to cross-tabs. One is to assume that even some pretty big sub-group numbers are as good as the overall poll. Headline poll ratings are carefully weighted to reflect the national picture, while the regional breaks are not. They bounce around a lot; London sub-samples in the regular YouGov polls varies between numbers that seem to imply a pro-Tory swing since 2010 to some with big Labour leads. But one can still use these figures cautiously (because the pollsters don’t guarantee that it will work, even if the numbers look sensible). If you want to know what’s going on in a subsection of the UK, the only safe ground is polling that specifically tries to do this. There’s reasonably frequent data in Scotland, and – a welcome development – Wales checks in every couple of months as well. Sometimes there are London polls. But the information on the different regions of England is sparse, largely because of the decline of the regional media, and for what there is – often linked to clusters of marginal constituencies – we owe thanks to Lord Ashcroft.

For what it’s worth, the swing to Labour in Scotland is low or negative (Labour did well in 2010 and the Scottish Tory vote seems to have bottomed out), and there is little significant differentiation in swing between Wales and most regions of England. The exception is London, which seems to have a low swing (again, perhaps a correction for Labour’s relatively robust performance in 2010). But local elections are probably a more reliable guide than poll subsamples, and they have tended to show a more regionalised pattern of swing with the north swinging harder to Labour than the south, and the Midlands varying. The May 2014 elections will generate lots of information, at least in urban England.

The other temptation of tables of polling data is to look at a small subsample and think it says anything at all; you can’t say anything at all about what east Midlands Lib Dem voters think about the bedroom tax, for example. ComRes has a nice little margin of error calculator here – a subsample size of 50 will have an error margin of 13.9 percentage points.  The human mind has a tendency to see patterns and significance where there is none – it’s called apophenia. When downloading poll details, remember the slogan: ‘avoid psephological apophenia’ – although I can’t see it catching on.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses cookies. Find out more about this site’s cookies.