This is the first in a series of 3 blogs providing some tips on questionnaire design. These principles are equally applicable to students, in the workplace, or even for a school assignment! Questionnaires are very popular tools to gather data in a wide variety of research projects. Some seem to think that “anyone can design a questionnaire” and that putting a few questions together which relates to your topic without proper planning and scrutiny will give you the answers that you need. Later on in the project, it often appears that this questionnaire has neither validity not reliability and actually gives you very little information. The popularity of questionnaires is due to a variety of reasons which include the possibility to reach a larger sample than with interviews, for example, as well as the relatively low cost. The accessibility of online survey platforms, as well as the possibility of social media platforms, have made the process of distributing questionnaires and gathering data even easier than before. The fact that the process is easy, does not mean that it is easy to design a questionnaire! In this first, of a series of 3 blogs, I will discuss some general principles of questionnaire design which are often overlooked in my experience.
- Keep the questionnaire short. This may seem obvious. When you are completing a questionnaire yourself, you prefer it to be short and may lose interest when it becomes too long. However, it appears that this principle is not always adhered to when designing a questionnaire to be completed by others! It is an art to find a balance between getting the information you want and keeping the questionnaire short and to the point. To this end, only ask questions which add value. A shotgun approach of “while I am at it, let me ask this as well……” is not effective. Beware of questions which are essentially asking the same thing, just for the sake of having more questions. Be sure each question is making a unique contribution and is clearly linked to your research questions.
- BUT at the same time: Have more than one question per construct. If you are measuring latent constructs (i.e. things you cannot see, such as job satisfaction, engagement, personal meaning, personality, wellbeing, attitudes and the like) the only way you can obtain a valid and reliable measure is to measure it with more than one question – at least 4-5. If you are measuring knowledge, this is not necessary.
- Avoid questions with which almost everyone would agree or disagree. The principle of correlation is covariance – put in simple terms, in order to test relationships between variables there needs to be differences in responses. If almost everyone gives the same answer, it adds little value to any further analyses. Questions such as the following may have very few people disagreeing:
“Technology makes life easier”
“Every company needs an effective cyber-security system”
“All people deserve to be treated with dignity”.
- Avoid a middle option on the response scale. The “classic” Likert scale usually has 5 scale points, of which the middle option are usually named things like “neither agree nor disagree” , “neutral”, etc. To know that someone has no clear opinion on a matter adds very little value to your study. Forcing respondents to at least choose to one side or the other, makes for more meaningful data. A 4- or 6 point scale would serve this purpose better. Again, a wider response scale makes for better variance.
- Related to this is the issue of response options. While some form of agreement with a statement is often used, many questions may be better addressed by a measure of frequency – people tend to find it easier to respond to this.
For example :
How frequently does your supervisor provide you with feedback on your performance?
Never / Seldom /Sometime / Often / Very often.
Another option would be to ask about the extent to which a certain behaviour is observed
To which extent does your company make use of an agile project management approach?
Not at all / To some extent / To a moderate extent/ To a large extent
If such a scale is used, it is important not to have an indication of frequency in the question itself, for example: “My team always meet their targets”. The indication of frequency should come from the response scale.
More principles of questionnaire design will be discussed in the next blog.