Once the purpose and focus of your evaluation are defined, you will need to consider your overall approach and how this will drive your choice of methods
The first step is to define the purpose(s) for the evaluation:
This will be followed by defining the focus for the evaluation and the key questions you would like the methods to address. For example:
As a general rule, the evaluation focus should be aligned with the targeted learning outcomes for the course – what you set out to achieve with the initial design of the course - and consequently linked to the overarching course objectives.
Once you have defined the focus, you will need to consider your overall approach and how this will drive your choice of methods. This will be underpinned by your research philosophy in terms of what you believe can be reliably captured through evaluation methods, what is worth evaluating and why this is the case.
Broadly, this might involve the following approaches:
An outcome-based approach might focus on the measurement of how far students have met the targeted learning behaviours and objectives. This might involve measuring levels of engagement (eg time on task / number of visits to a VLE module site) and the effectiveness of the learning (eg performance in formative and summative assessment).
An interpretive study might focus more on staff and student perceptions of study methods and how they are understood and valued. This could be achieved by inviting students to recount their experiences of learning in their own words, elaborating on the context of their learning and the link between formal and informal study methods. Attention would be drawn to affective & attitudinal variables (learners’ feelings and levels of motivation towards the approaches used), and the role of prior knowledge and experience in influencing their reception of the study methods. Staff could also be invited to provide their perceptions of students’ learning relative to previous performance or previous cohorts?
In case study evaluation these approaches can be combined to generate a multi-dimensional picture of the learning that has taken place. For example, qualitative techniques are used to capture data on students’ perceptions and feelings on how the course has unfolded, while quantitative techniques generate data on what students know and what they do.