More on When to Avoid Asking Direct Questions in Market Research

Previous research on voting intention at the 2017 UK general election strongly suggested that we’d be better off asking research participants for estimates of other people’s behaviour, rather than their own. Although initially counter-intuitive, this is backed up by what behavioural science tells us about our inherent psychological biases. Intrigued by the simplicity of the idea, and the number of its potential applications, we set about doing more work – this time in our home sector of healthcare.

To help establish whether the technique worked, we again looked for hard outcomes against which to measure, as we had with the general election result. We found this in the form of the ABPI’s “Disclosure UK” initiative, which publishes payments made by pharmaceutical companies to health professionals (in return for their consultancy in promotional, educational, and/or R&D activities). The initiative is not perfect – it does not oblige health professionals to agree to disclose such payments, but about 70% do so. We know from these and other publicly available data that ~22% of UK hospital doctors who agree to take part in the initiative, declare receiving such payments. If we surmise that of the 30% who do not agree, most decline because they are shy about disclosing payments they have received, we can conservatively forecast that at least 35% of all UK hospital doctors receive such payments.

The above shows that, when we asked n=94 UK hospital doctors (a mix of gastroenterologists and haematologists) either a) directly about their disclosures (n=45), or b) indirectly about a colleague’s disclosures (n=49), we saw differing outcomes. The group asked directly were less likely to declare having received such payments (27%), whereas the group asked to speculate about the behaviour of others estimated substantially higher (35%) – a figure much closer to our conservative estimate of at least 35%, based on the real world data.

There is a bunch of established behavioural science that says we shouldn’t be in the list bit surprised by this. So, if we tend to be more honest when the subject is other than ourselves, shouldn’t we ask our research questions accordingly? Indeed, the main reason I’m pursuing this line of thinking in these blog posts is that it is everyday apparent to me – via the day job – that within the market research industry the direct question still holds sway.  I suspect that the idea of asking indirectly fills some researchers and clients with dread. Not only would it be a break with the status quo, it has a certain discomfit attached – are we really that bad at estimating our own behaviour? This is yet more evidence that, yes, we are.



Leave a Reply

Your email address will not be published. Required fields are marked *