Creating course evaluation surveys is an important part of assessing the learning experience of your students, and it’s important to design the survey in a way that will provide accurate and useful feedback. One of the major challenges in creating a successful course evaluation survey is avoiding question-related biases that can distort results.
Here are the best practices for designing your course evaluation survey without compromising accuracy. What are on of the most common pitfalls?
Less is more: shorten the list
Non-response Bias is when respondents are more or less likely to respond to certain types of questions than others, or fail to answer a question entirely. To avoid this type of bias, try reducing the number of questions in your survey.
Instructors should try limiting the use of open-ended questions, and making sure the questions are easy to understand and concise. You should also consider including an incentive (such as a discount for completing the survey) to encourage respondents to answer all the questions.
Maintain neutrality is key
Acquiescence bias happens when respondents are more likely to agree with a statement than disagree with it. Neutrality is the best remedy in this case. For example:
- Acquiescence bias: Do you agree that you have made significant progress on your work this week?
- Non-biased: What progress have you made on your work this week?
All the questions in the survey need to be worded in a neutral way and by avoid leading questions. Additionally, the options provided in the survey should be mutually exclusive to ensure that the respondent actually has the possibility of disagreeing with a statement.
Set respondents free with anonymity
Social Desirability bias occurs when respondents answer in a way that conforms to the cultural norms, regardless of what their own thoughts and opinions actually are. To reduce this bias, you should keep the survey anonymous and remind respondents that their honest opinions are welcomed and appreciated.
You should also avoid using language that implies judgment, such as ‘good’ or ‘bad’, and try to include at least one open-ended question where respondents can freely express their true feelings. For instance:
- Social desirability bias: Aren’t you happy with the job you are doing?
- Non-biased: How do you feel about the job you are doing?
Randomize answers and questions
Question order bias can occur when the order in which the questions are asked can influence the answers provided. To prevent this, try to keep the sequence of the questions as random as possible.
Avoid asking multiple questions on the same topic in sequence as this could lead to respondents basing their answers on the previous question. Additionally, you should list the response items in a random order whenever possible, instead of using the same response options in every question.
Bias can cause students to feel uncomfortable and lead them to respond with answers they don’t truly believe. This invalidates the responses, making the survey data unreliable. By using questions that are more neutral, the survey results can be more accurately represented, giving a truer picture of student satisfaction and the effectiveness of the course.
Crucially, asking unbiased questions promote a feeling of less scrutiny for respondents, which leads to more authentic responses. By ensuring survey questions are unbiased, valuable insights can be gained that can inform future course programming.