LEARN! Manual Example #25

This example shows what methods the Danish Road Safety Council uses to continuously evaluate their “Road Safety LIVE” initiative and what results it found.

Related Step

8

Evaluation

Age Group

13 to 16 years old

Evaluate. Execute your evaluation plan by conducting process and outcome evaluations. Ensure the quality and relevance of your activity over time, and write a final report.

In addition to the randomised controlled trial to evaluate the effectiveness of the “Sikker Trafik LIVE” (“Road Safety LIVE”) initiative, as presented in example 24, the Danish Road Safety Council also continuously conducts post-activity measurements to evaluate whether the activity is achieving its objectives (see example 4). For these post-activity measurements, the council uses the following quantitative and qualitative methods:

  • Quantitative: Survey (mobile survey answered via smartphones) for pupils, who answer the survey just after they have been taught in the classroom. (See example 26 for the survey).
  • Quantitative: Survey (mobile survey) for teachers, who receive the survey just after the visit. (See example 26 for the survey).
  • Quantitative: Survey (mobile survey) for pupils. (See example 26 for the survey). This follow-up survey is send to the pupil´s mobile phone either 6 to 10 months or 12 to 16 months after they have participated in a LIVE visit. The reason for the difference in when they receive the follow-up survey was to see what the longer-term effects of the activity were. So far, the analysis of the results shows that the effect mostly remained, but fewer pupils answered the survey when they received the follow-up survey 12 to 16 months after the visit. Consequently, the results must be seen in the light of this. From 2021 onwards, follow-up surveys will therefore only be sent out 6 to 12 months after the visit.
  • Qualitative: Observation of visits and interviews with pupils a couple of times each year.

Having used these different evaluation methods, the Danish Road Safety Council was able to determine that 89% of the pupils knew which the most vulnerable age group in traffic was during the 2017/2018, 2018/2019 and 2019/2020 school years. Moreover, their evaluation showed that during those three school years, respectively 99%, 98% and 98% of the pupils rated the initiative as good or very good.

An ambassador at a Road Safety LIVE visit. Source: Danish Road Safety Council.

The results on these objectives together with the achieved reach – 36,612 pupils in 2017/2018, 36,763 in 2018/2019, and 26,185 in 2019/2020 – were used as metrics reported to the foundation that supported the council in implementing the LIVE initiative. (The amount of pupils reached during the school year 2019/2020 was lower than usual due to COVID-19 restrictions.)

The results on the behavioural objective were positive as well, with 94% of pupils in both 2017/2018 and 2018/2019 indicating that they had now taken better care of themselves in traffic as a result of the LIVE visit.

The results of the evaluation of the knowledge objective via a survey undertaken 6 to 12 months following the LIVE visit showed that 86% of the pupils in 2017/2018, 87% in 2018/2019, and 85% in 2019/2020 still knew that they belonged to the age group that is most vulnerable in traffic.

Coupled with the data from the quantitative surveys, the observations and the interviews show how the individual ambassadors are performing. Do they give the pupils the knowledge that is set out in the theory of change one-pager for the “Road Safety LIVE” initiative? Does their education live up to the 5E models’ criteria? Is their message clear?

The evaluation of the Road Safety LIVE initiative was presented during the LEARN! Manual Webinar.


More Information


Related LEARN! Material

Categories: