Pretesting and revising a questionnaire

When they are first drafted, questionnaires often contain questions that are ambivalent, cumbersome and vague. Instructions may also be confusing: the questionnaire may be too long and questions that should have been included may have been omitted. The pretest is a means of discovering the faults in a questionnaire before it is administered. To pretest a questionnaire, a small subsample of the intended respondent group is selected: perhaps a dozen or so people. A good range of respondents is needed. The pretest can be run as a debriefing session or by using the protocol method.

Debriefing method

Here the questionnaire is presented to respondents in exactly the same manner as it would in a real study. After completing the questionnaire, respondents are asked about their thought processes while they were completing it and whether there were any problems with such items as routeing and branching instructions, understanding questions, and so on.

Protocol method

This involves respondents thinking aloud in the questionnaire (protocol analysis).

In examining the findings of a pretest, ensuring that:

as the interview proceeds or as they fill particular attention should be given to

• The meaning of the questions is clear.

• The questions are easy to answer.

• The questions flow logically from one to another.

• The routeing/branching instructions are clear.

• The questionnaire is not too long.

• The questionnaire engages and retains the respondent's interest.

There are at least five considerations to be taken into account when conducting a pretest of the questionnaire:

1 All aspects of the questionnaire should be included, most specifically:

• layout

• question sequence

• word meaning

• question difficulty

• branching instructions.

2 The pretest should be administered and conducted in an environment and context that is identical to the one to be used in the final survey.

3 The majority of the pretest interviews should be undertaken by regular staff interviewers. This should include experienced and relatively new interviewers.

4 Respondents in the pretest should resemble and be representative of the target population. This means that they should be familiar with the topic and possess similar characteristics, attitudes and opinions to those in the target population.

5 The size of the sample used for pretesting will depend on the variation of the target population. The more heterogeneous the target population, the larger the pretest sample required. And the more complex the questionnaire is, the larger the pretest sample should be.

Was this article helpful?