Surveys are an inescapable part of our society. Hardly a day passes without a mention in the media that “A survey has revealed such and such”. If you’ve managed to get through life without ever being approached by someone with a clipboard asking you to answer a few questions, or being called up, or written to, at home asking you to take part in some survey research, well, you’re a pretty unusual person. But can surveys really tell us anything useful?

Okay, suppose I want to know how many people in the UK enjoy eating mangos. I could ask everyone, but that would take a very long time, and in any case it’s not necessary. If I make some stew, and I want to know how it tastes, I don’t have to eat the whole saucepanful to find out. I taste it - that is, I take a sample. For this to work, however, I have to do it properly. It would be no good just taking a spoonful from the top, without stirring first. I’d get the herbs that had floated to the top, and miss the onions that had burnt onto the bottom. I want my spoonful to be representative of the whole panful, so I stir it up first. It would also be no good to eat a salty biscuit and taste the stew straight afterwards without rinsing my mouth out. The salt in the biscuit would bias my perception of the taste of the stew.

To answer my mango question, then, I don’t have to ask all 60 million people in Britain. I just have to ask some of them. But my sample of people has to be representative of the whole UK population, in terms of mango-eating habits at least. And I have to ask them about mangos in an unbiased way — it would be no good saying “What do you think of these delicious and health-giving fruits?”, for instance.

The art of carrying out a good survey is a complicated one, and involves compromises and trade-offs as well as scientific and statistical principles. For instance, the best approach to choosing a sample of people for a survey is usually to choose them at random. (In a way, this is the statistical equivalent of stirring the stew before tasting it.) But there are many ways to do that. In my mango survey, I could choose a random sample of people from right across the UK. But they would be spread many miles apart from one another, and in going round the country to interview them, I’d spend most of the time travelling. It would be better if I could group my interviews together into a few towns, and then interview several people in each town. Other things being equal, that would make my results less accurate, because the towns I pick might not be representative, even if I choose them at random. On the other hand, by grouping my interviews into towns, I could reduce my travelling costs, so I could afford to do a lot more interviews, and the gain in accuracy from these extra interviews could well outweigh the loss in accuracy from clustering the interviews in towns.

    next > Page 1 of 2

Comments