LEARN! Manual Example #12
This example shows how the Danish Road Safety Council pre-tested their new version of the cycling test for Grades 5 and 6 in two stages.
Related Step
Pre-Testing
Age Group
12 to 13 year olds
Before a new version of the Danish Road Safety Council’s bicycle test was pre-tested, the “old” cycling test had already been tested and evaluated in several ways, both quantitatively and qualitatively. These tests showed the need to upgrade the test. Consequently, a new test was developed, which included both in-class education (theory) and out-of-class education (practice) on roads near the school, including a 1.5-5 km route the pupils would have to ride.
The new cycling test was designed to include the 5E model:
- Engage: the test includes activities that motivate and activate the pupils
- Explore: the pupils explore and collect information by themselves
- Explain: the test provides (theoretical) knowledge on safe cycling
- Elaborate: the acquired theoretical knowledge is used in practice
- Evaluate: the pupils evaluate their own performance and that of their classmates during the theoretical and practical test. Furthermore, their performance is evaluated and discussed with the teacher after both parts of the test.
The 5E model
The pre-testing of the new test was done in two stages.
Stage 1: Testing the concept
A qualitative method was used to test the concept, and included two focus groups: one in east Denmark, one in west Denmark. Each focus group consisted of 12 teachers, and therefore 24 teachers were involved in the test in total. The teachers represented different types of schools (based on a division between city and rural schools, and socio-economic characteristics). Half of the teachers had taught the old cycling test. All teachers received a gift card worth 50 Euro for participating.
Each focus group lasted three hours and was led by a moderator. The focus groups also included another person, who via a PowerPoint presentation showed the participants what had been developed so far and what was intended to be included in the new cycling test and its teacher´s guide. The participating teachers were asked different questions related to the topics listed below. Before the discussions started, the teachers had to individually fill out a paper with different questions related to the topics. This was done to avoid them being influenced by the others and to make them reflect on the new test and its teacher´s guide. The topics discussed during the focus groups included:
- What was considered important to include in the teacher´s guide, with regard to the description on how the teachers should teach the activity to the pupils. (The teacher’s guide had not been developed at that time, nor had the information on the website with teacher´s pages and pupil´s pages);
- The theoretical part of the new test. This included the theoretical test itself, the PowerPoint presentation on theory that the teacher should use to teach, and the pupils’ group work regarding the theory;
- The practical part of the new test. This included checking whether the bicycles are up to safety standards, pupils cycling the route in traffic, pupils and adults checking other pupils cycling the route, and the follow-up in the classroom.
Other issues discussed included whether the teachers expected pupils to attain the envisaged knowledge and experience as intended, whether the teachers would use the new test, and whether they would recommend it to others.
After the focus groups had been held, a report (in PowerPoint format) on the results was presented and changes to the concept of the new cycling test were made. Among the report’s conclusions were that the teachers would use the new cycling test, and that they especially liked that the test had been updated with a new design, that it had been simplified, that it had a better connection between the theoretical and practical parts, that there was more focus on having the pupils reflect on their own and others’ behaviour on bicycles, that it included more group work, and that it focused on cooperative learning practices.
Input received from the teachers included that they felt that the teacher’s guide should also be easy to use for teachers who had not previously taught the (old version of the) cycling test, that the tests should be usable on different devices and formats (e.g. on PCs, tablets, Chromebooks, etc), that it should be clarified which national objectives/goals were being fulfilled, that the answers to the questions in the pupils’ material should be included in the teacher’s guide, that the pupils’ pages should be easy to read and understand for participants who do not master Danish perfectly, and that participants should be able to write directly in the PDF files.
Pupils participating in the Danish cycling test. Source: Danish Road Safety Council.
Stage 2: Testing a “mock-up/draft” of the teaching material, the website and teacher´s guide
The next stage was to test the new cycling test in practice to see how pupils and teachers received it.
A mix of quantitative and qualitative testing methods were used:
- Quantitative: Survey (mobile survey filled in via smartphones) for teachers regarding the theoretical in-class education, filled in by the teachers just after they had given the theoretical lesson;
- Quantitative: Survey (mobile survey) for teachers regarding the practical out-of-class education, answered by the teachers just after the practical lesson;
- Quantitative: Survey (mobile survey) for pupils regarding the theoretical in-class education, answered by the pupils just after they had been taught;
- Quantitative: Survey (mobile survey) for pupils regarding the practical out-of-class education, answered by the pupils just after the execution;
- Qualitative: Observations of the theoretical in-class education and the practical out-of-class education; and
- Qualitative: Short interviews with teachers and approximately 6-8 pupils from each class after they had finished the test.
Among the areas covered were knowledge and competences gained by the pupils in both the theoretical and practical parts (did they acquire the competences to cycle safely alone in traffic?), functionality, liking, relevance, reflection, intentional behaviour, etc. The analysis of the surveys and the interviews was conducted by documentation specialists. They also conducted the observations, but here the developers of the new test also participated to gain relevant insights. The 5E model’s points mentioned previously were also evaluated via the observations, interviews and surveys.
This test was done at six different schools in Denmark, again representing different types of schools (based on geography, a division between city and rural schools, and socio-economic characteristics). Some schools had previously used the old cycling test, while others had not. The teachers each received a gift card worth 50 Euro for participating, while their pupils received an ice cream. The test took approximately five hours at each school.
After the test was conducted, a report (in PowerPoint format) on the results was presented, which concluded among other things:
- On the teacher’s guide: it should have a simpler layout so it is easier for the teachers to get an overview and know what to do step-by-step for the preparation, the theory part, and the practical part (both the bicycle checks and the test). It should also provide background to the cycling test (Why is the new cycling test being introduced?, What are the national goals and key performance indicators?, etc).
- On the theoretical part: the introduction film and the theory test at the beginning both worked well; however the theoretical education was too long for the pupils and therefore required changes. The group work also required changes in order to function better.
- On the practical part: the teacher’s guide should highlight how to start up and frame the day and what the pupils should do. The bicycle checks done by the pupils and supervised by one or more adults were found to be working well, and while the practical cycling test in traffic also worked, it could be improved by giving the adults a better introduction to their role
Changes to new cycling test were made, and it was subsequently launched for all the schools in Denmark to use. The use of the new test is monitored (as log-in to the website is required) and shows a high usage. The evaluation of the new test has been scheduled to take place approximately 1-1.5 years after the launch.
More Information
The teacher’s guide for the new cycling test (in Danish).
LEARN! Manual Example #12: Pre-Testing a New Version of a Cycling Test