Market research tells its participants up-front that they, “have the right to withdraw at any time”. Depressingly, this prerogative is being exercised with a frequency that continues to reduce the numbers willing to repeat their research experience. We know that the primary culprit is the ubiquitous online survey, but the degenerative process affects market research generally, slowly undermining the credibility of our insights and, perhaps, even our business intelligence role itself.
In healthcare, the problem is especially acute, because our sample universes are often small and specialist to start with, and because we have come to rely on surveys to keep insight wheels turning:
- Online surveys represent about 75% of healthcare quantitative market research;
- 95% of the health professionals who do market research, participate in online surveys;
- On average, health professionals who take online surveys do so at least once per month.
Good practice is being undermined
A poor online survey experience causes serious collateral damage on two fronts:
|To the project:||To sample quality:|
|Bad experiences lead to frustrated participants, who are more likely to give bad data, that in turn gets used to generate bad insights.||A poor experience is less likely to be repeated, making it harder to recruit the right people, which in turn makes fieldwork slower and more expensive, and over time leads to less representative samples.|
The BHBIA’s 2017 Response Rate report helped us understand what most irks healthcare professionals about online surveys. Here are their top three complaints:
- Failure to qualify through “screening” questions;
- Lack of trust, in our processes and professionalism;
- Poor survey design, presentation, and usability.
Sadly, serious researchers who care about being open and inclusive, and authoring thoughtful online questionnaires are being crowded out by non-researchers willing to deliver at the lowest possible price point. In this cost-saving climate, research buyers and internal procurement departments seem magnetically attracted to these cheaper and often over-promised propositions that tend to treat participating health professionals more like commodities than customers.
Certainly, the BHBIA’s response rate report helped raise awareness and usher in some sensible new guidelines and processes. Yet we continue to see approved questionnaires inspired more by forecasting spreadsheets than humanity, failed programming, inept communications, unfulfilled samples, and so on. It feels like a race-to-the-bottom in which we are all participating, unwittingly or otherwise.
Time to work together
Having said all this, I admit that too much is based on personal experience and the testimony of colleagues rather than hard fact. Regrettably, the panel companies who, with their recruitment partners, administer most online fieldwork, do not publish response rate trends. Presumably because they do not wish an unpalatable truth to be made public. Equally, we do not yet have a standard metric for survey quality, only individual companies asking about satisfaction in myriad ways.
What can we do?
In my view there are two specific things that would get us started:
- Demand response rate data, anonymised if necessary, so we can say for sure how much trouble we’re in;
- Establish an industry standard metric for the online survey experience.
However, whether we are consumers or creators of online market research the most important thing is that we start to promote and buy based on quality, not price. Obviously, this is easier said than done, but if we don’t try to interrupt the current fashion for false economy, we will eventually run out of participants, time, and credibility.