Within our alliance we have several CBL courses that have their own target audience, design, topic domain and way of working. The IDC team developed a procedure for evaluating CBL education and the courses provided by our alliance. Following this procedure will help you evaluate different aspects of your course and improve it continuously.
New courses, especially, will benefit from organizing focus groups and/or interviews with all involved parties to evaluate the process and the collaboration throughout the organisation of the course. Evaluation should begin at an early stage, not – as often believed – after completing the course. Constant evaluation allows for ad-hoc improvements and can increase the feeling of ownership for teachers, coaches, and learners. In CBL education, the learner takes the lead. This asks for a different approach to evaluation. Ideally, the core team, coaches and learners have joint responsibility for the quality of the learning process and the end product. In our courses, the core team and mainly the coordinator are responsible to set and organise evaluation moments throughout the course.
Before the start of a course, the core teams needs to determine the evaluation process, with the help of a member of the IDC team. The figure below shows possible building blocks for evaluating EWUU courses based on evaluations of previous courses.
Input for evaluation can be collected in several ways: (online) surveys, focus groups, informal chats, online voting tools such as MentiMeter, etc. Several topics should be included in your evaluation, such as intended learning outcomes, expectations, teaching-learning environment, roles and responsibilities, assessment and learning outcomes achieved to date. These topics do not necessarily apply to all parties.
Learner feedback can formal and informal. Formal online surveys can be sent out to learners at the beginning of the course, throughout the duration of the course with pre-planned feedback moments, and at the end with a final survey. Collecting feedback at these different moments provides insight into how the course is progressing and experienced by learners, and potential changes/improvements to the course.
Initial feedback should concern initial expectations, the registration process and depending on when the feedback occurs, first impressions; all three can provide useful insights around getting started with CBL at EWUU.
At the end of a course, final feedback can be collected through a survey which would tend to be longer with more in-depth questions. As and when needed, short feedback moments such as interviews or focus groups with learners, coaches and core team members can be scheduled to discuss issues arising or, simply, experiences to date. These help evaluate the process and organisation: minor alterations can be made where necessary.
Close to the course start date an introductory meeting with coaches is recommended to discuss expectations on both sides. Furthermore, conversations with coaches can give insight into how the challenge is going and their experience with learners.
It is recommended to organise alignment meetings for the different coaches of a course in which experiences and feedback can be shared.
Input provided by the challenge agent is significant, as they are the main actor of the challenge with an essential role in advising the learners. Challenge agent feedback/input can occur in similar ways to those mentioned above, formally and informally for the length of the course. Topics can include organising the course, the experience of being a challenge agent, reviewing their expectations, learner deliverables and their connection with the learners.
Feedback from experts would be similar to that detailed above (informal conversation and/or a survey designed for experts and stakeholders). Feedback issues would include organisation, teaching-learning environment, assessment, learning outcomes, and the overall process. Again, a final conversation is recommended, with particular focus on educational framing, and the content and form of the learning activities. Topics can include: how did you experience your session, could learners follow the session, did learner miss any foreknowledge, did learners show interest in topics outside of your expertise, how was the engagement within the group, etc.
The course organisation process and the collaboration should also be evaluated by the course coordinators. A member of the IDC team can join an evaluation meeting. Moreover, the IDC team organises meetings every half year with members from all core teams. The aim is for the core teams to exchange experiences. IDC team members answer questions about the organisation and act as moderators. .
Evaluation results
Course evaluation results should be provided to your linking pin of the IDC team. This can also occur as a face-to-face meeting with a member of the IDC team. One of the goals is to share lessons learned and best practices across the whole organisation.
Changes derived from evaluation results can be implemented while the course is active. This (single-loop learning) enables the courses to be improved and problems solved. Double-loop learning – implementing the changes in a following course – helps identify patterns and ongoing insight into the course process and issues.
See the resource section for templates and tips and tricks for course evaluation.