This is the last of a series of 3 blogs containing tips for questionnaire design. You can find the previous 2 here:
9) An issue that remains a contentious one in questionnaire design is that of positively vs negatively phrased questions. The theoretical ideal is that a questionnaire should contain both positively phrased questions (where agreeing with a statement reflects a positive attitude / high importance / strong presence of the aspect in question) and negatively phrased questions (where agreeing with the statement reflects the opposite). The argument is that this will avoid so-called response set – the tendency to answer all questions in more or less the same manner, regardless of their content.
- I feel energised when I wake up in the morning (agreeing denotes a high amount of energy – positively phrased)
- I struggle to get out of bed most mornings (agreeing denotes a low amount of energy – negatively phrased).
In order to calculate a composite score where a high score denotes a high energy level, you will have to reverse score the negatively phrased question. This is done after the data has been collected.
I find more and more that people are lazy to read, and even more lazy to think when completing questionnaires. This is especially true when they have no vested interest in completing the questionnaire (there is “nothing in it” for them), or when they are in a hurry and just want to get it over and done with (which is quite common). There is often some survey fatigue in, for example, student populations who are often targeted for research purposes, or in companies where a lot of surveys are performed.
How does one get around this? There is not one clear answer to this. Phrasing all questions positively (or negatively for that matter) may lead to response set as mentioned above, which may, in turn lead to very little variance in answers see Blog 5 Tips for questionnaire design (1).
One useful tip is to try to avoid using the word “not” in negatively phrased questions. For example:
Positively phrased question:
- My supervisor evaluates my performance fairly.
A negatively phrased version of this question could be:
- My supervisor does not evaluate my performance fairly.
- My supervisor is biased when evaluating my performance.
When a respondent has to agree or disagree with these statements to a certain extent, it takes an additional cognitive step to think: “I do not agree that my supervisor does not evaluate my performance fairly”. Phew! That is a lot of thinking to do!
When a negative word such as “biased” is use in stead of “not” this cognitive step is not necessary and the chances are better that you will get a meaningful answer.
10) When a questionnaire is developed, it is meaningful to arrange questions according to the topic it addresses, often with a heading indicating this topic. While this is useful in the development and review phase, including this in the questionnaire provided to respondents may lead to response set as indicated above. – where respondents tend to answer questions in a similar fashion, regardless of the questions. Seeing a heading such as “Remuneration” may lead a respondent to answer according to their general attitude towards remuneration, without consideration to each individual question. The same may go for sections like “Leadership”, “Performance management” and the like. The solution to this is to scramble the questions when presented to the respondent without any headings – in this way one may avoid biased answers, simply based on the collective theme of the set of questions. Afterwards, the items can be grouped together according to themes.