#12 – Undertake tests, process and/or outcome evaluations
Evaluations should be a central part of any traffic safety and mobility education intervention.
Evaluations should be part of the plan from the intervention’s inception and should be conducted properly. The evaluation results of other interventions should furthermore be taken into account during the design of an intervention.
While evaluations should be part of the intervention, the scale of the evaluation should be proportionate to the scale of the activity. The results of the evaluation should also be used to improve the intervention itself.
The LEARN! Manual provides guidance, recommendations and inspiration on evaluating educational activities on traffic safety and mobility, notably in Step 8 of the Manual’s model for developing and evaluating educational material. The chapter in the Manual covers:
- Why it is important to properly evaluate activities.
- The different types of evaluations.
- Performing outcome and process evaluations.
- The importance of measuring both output and outcomes.
- Ideal requirements for outcome evaluations.
The Manual also contains several examples setting out how members of the LEARN! Expert Panel evaluate their material. These examples are available in the LEARN! Database of best practice examples.
BEST PRACTICE EXAMPLES
Scotland – Independent Evaluation as a Criteria for Funding
The Road Safety Framework Fund is designed to promote and encourage further partnership to ensure the delivery of road safety commitments in Scotland, including through education. There are two rounds of funding each year.
The scheme developers can bid for funding and, as part of that, they must include plans for independent evaluation. There is also an evaluation fund which allows those who have been running schemes for two years or more to commission an independent evaluation.
Further information on Scotland’s Road Safety Framework Fund can be accessed here.
Denmark – An Overview of Different Evaluation Methods
All the teaching materials provided free-of-charge to the schools by the Danish Road Safety Council (Sikker Trafik) have been evaluated to make sure that they meet the national goals for traffic safety education and have the desired effect.
As a minimum, the evaluations include an online mobile survey to be filled in by pupils immediately after the lesson. The pupils have to send a short text (for example “traffic”) to a number, after which they receive a link to a survey which they can answer on their phone. The surveys usually include questions on liking (what they liked/disliked), what they have learned (new knowledge and reflection), what their (former) traffic behaviour was and what their intended behaviour in traffic is after the lesson.
Other evaluation methods include similar online mobile surveys for teachers, observations of the teaching materials being used by teachers at different types of schools (geographically, public/private, socioeconomic), and interviews with pupils and teachers.
All new and revised teaching materials are (pre)tested. Focus groups are conducted with teachers from different types of schools, where they are introduced to the material and the teacher´s guidelines, and adjustments are made based on the gathered inputs and insights. The material is subsequently tested at different schools using the same methods previously mentioned for the evaluations. All tests and evaluations are conducted by the Danish Road Safety Council’s evaluation experts.
The Danish Centre for Social Science Research has also conducted a randomised controlled trial (RCT) study of the Danish Road Safety Council’s LIVE ambassador visits, a programme for schools with pupils aged 13-25 years. Half of the schools in the study received LIVE ambassador visits while the other half (the control schools) did not. The study showed the effect on the pupils´ knowledge, attitudes, intended behaviour and actual behaviour. While this kind of effect evaluation is “state of the art”, it is also time consuming, expensive and requires the involvement of a qualified research centre. Valuable insights and knowledge can nevertheless also be achieved with fewer resources when using the other described evaluation methods.
Documents on the results of the study are available in English.
The best practice examples are not yet available on the mobile version of the website. Please visit the website on a tablet, desktop or laptop to see them.