Making sense of your evaluation data and taking action
Interpreting the data (addressing potential challenges)
Reflection in action (in order to make adjustments as the module progresses)
Reflection on action (in order to adjust the design for the next iteration of the module)
Interpreting the data:
- Recognising and mitigating for challenges including:
- Engagement (eg survey fatigue or motivation to engage in focus groups). This can lead to challenges in a achieving a representative sample of a cohort for focus groups. For example, volunteers may reflect the most motivated students within the class or those with strong positive or negative feelings about particular approaches, presenting a one-sided view of the learning experience. For international cohorts, cultural issues may come into play, with some groups of students reluctant to offer negative feedback on the course design or level of instructional support for their learning.
- Visibility and context (data collected may not be comprehensive enough to capture the full range of learning in formal as well as informal settings)
- Halo/horns effects (eg Impact of assessment outcomes for individual responses or other potentially biasing factors relating to perceptions of teaching staff).
- Triangulation of results with other data sources may help to confirm or throw into doubt the trends that have been identified.
Reflection in action
- Formative evaluation focuses on processes as well as outcomes so will allow some adjustments to be made as the module progresses (depending on timing and nature of measures to uncover these).
Reflection on action
- After the module, reviewing the data in the light of the original goals should allow for adjustments to be made (eg design, activities, materials, instructions, feedback and support).