Personally, I can’t wait for the UK general election on the 8th June. Not for the politics, but for the polling. And also to see if I’m right about this…
A week or so ago I fielded a couple of questions on Google Surveys, asking about voting intention. Not because I long to be the next Nate Silver, but because I wanted to gauge the size of a framing effect. Specifically, the difference it makes when asking about the participant’s own voting intentions, versus asking them to predict the voting intentions of a friend.
My intuition, based on lessons learned from psychology and behavioural science, was that we’d get a more “honest” aggregate answer when asking about a friend. A summary of the key differences are shown in the table below. I’ve deliberately kept politics out of it!
When participants answer about themselves, we see virtually no difference between Labour and Conservative shares, but when asking about a friend we see the Conservative figure almost three percentage points ahead. All the major polls put the Tories well ahead, and I freely admit to not having attempted any sophisticated weighting so I expected a (strong) Labour bias in the absolute levels reported. With those things in mind, the survey results do indeed show that we get the more ‘realistic’ answer when asking participants to answer in relation to a friend.
Intriguingly, participants express greater certainty when predicting how their friend will vote, than when asked how they themselves will vote! Perhaps this is in part because we fear giving ourselves away (even in an anonymous survey), but it also suggests to me that we think we know the minds of others better than we know our own!
Participants answering about a friend also return a more realistic turnout result – referencing the last half dozen general elections we can confidently expect an actual turnout figure less than 70%.
All of which suggests that participants are more authentic when asked to predict the actions of someone they know well, than when asked about their own intentions.
Many would say that voting intention is a sensitive topic and that many will not wish to divulge their true colours. In which case, adaptations of this technique may prove useful in obtaining better answers in other sensitive areas. But I’ll go one further and propose that this approach should work in pretty much any setting where judgement and opinion is being sought. For example:
- how do doctors rate the prospects for a new medicine?
- how will customers react to a new price?
- which advertisement would be most effective?
We needn’t ask about a ‘friend’ of course, it could just as easily be a colleague, a neighbour etc. – the important thing I think is that the participant knows the other person well.
Of course, it can feel uncomfortable to be asking about an unknown third party instead of our sampled participant, but I tend to think the bias is a benign one? And if the technique proves more accurate in these situations we should welcome it regardless, and set aside any squeamishness.
I’ll be running the above questions again in a few weeks time, for personal interest and – hopefully – to confirm the nature of the findings. I’ll let you know if they’re different! We have already used this approach occasionally in our own questionnaires, but the upcoming general election gives us an unmissable opportunity to test the idea definitively – against the outcome of the vote itself!