LEARN! Manual Example #14
This example shows the importance of designing and conducting the pre-test of an activity even when having only little time to do so, as it provides valuable insights into what is working and what is not, which then allows for the activity to be adjusted.
Related Step
Pre-Testing
Age Group
14 to 16 year olds
In 2018, the award-winning film “While We Live” was released, telling the story of a tragic traffic collision involving young people. The Danish Road Safety Council had co-funded the production with the condition that the film could be used free-of-charge by Danish schools via a password-protected website.
New teaching material therefore needed to be developed to accompany the film. It was decided that a teacher’s guide – with exercises and other activities to be used for teaching different subjects (traffic safety, social science, Danish, and religious education) – was to be developed and tested within only a few weeks, in order for it to be printed and sent to all the schools together with other materials. Consequently, there was only little time for testing and therefore the test design had to include what was best and possible under those conditions.
A qualitative testing method was used, involving two focus groups – one in east Denmark, one in west Denmark. Although the aim had been to get 12 teachers in each focus group (24 teachers in total), due to the teachers’ busy schedules, only eight showed up in the first focus group and four in the second. However, they represented different types of schools (including a division based on city and rural schools as well as socio-economic characteristics). The participating teachers each received a gift card worth 50 Euro.
The focus group with eight teachers lasted three hours, whereas the other lasted two and a half hours, and both were led by a moderator. The teachers were given a “mock-up”/draft of the teacher’s guide which already included all the text and pictures. They were first asked to look at the guide quickly – as if they had just received it at their school. They were then asked to write down their initial impressions of the guide.
The mock-up cover of the teacher’s guide as used during pre-testing. Source: Danish Road Safety Council.
They were subsequently asked to read the teacher’s guide and, while reading, to individually fill in a document (questionnaire) with questions related to the guide´s different parts. They were not allowed to speak during this part of the session, in order to prevent them from being influenced by the other participating teachers.
When the teachers had all read the guide and answered the questionnaire (which took approximately 45 to 60 minutes), the guide and the filled-in questionnaires were discussed in the focus group, with the discussion led by a moderator. The focus groups provided valuable insights into what worked and what did not, what was missing, what exercises the teachers would use and would not use, etc. These insights included:
- The structure of the teacher’s guide should be altered, with sections in a different order.
- While the film may be relevant for the subject ‘film education’, the exercises were not deemed to be relevant enough in relation to the subject. The subject should therefore be dropped from film education and instead included under the subject ‘social science’.
- More questions in relation to the statistical exercises should be included, as well as more analytical questions for when it was used for the subject ‘Danish’.
- It should have more focus on group work and cooperative learning, by including questions for the pupils which the teacher could print out and give to the pupils or show on a screen. More inspiration examples of different types of group work should also be given.
- The ‘reflection’ questions should be more precisely formulated and preferably refer to special places in the film where the relevant theme appears.
After the focus groups had been held, a report on the results was presented and changes to the teacher’s guide were made. It was then printed and sent to the schools. The use of the material is monitored (as log-in is required to access the film) and shows high usage. An evaluation of the material is scheduled to be conducted 1.5 to 2 years after the launch of material.
The final cover of the teacher’s guide, after the changes were made based on pre-testing feedback. Source: Danish Road Safety Council
More Information
“While We Live” teacher’s guide (in Danish).
“While We Live” teaching material (in Danish).
LEARN! Manual Example #10.2 While We Live.