LEARN! Manual Example #24
This example shows how professional researchers used a randomised controlled trial to evaluate the Danish Road Safety Council’s “Road Safety LIVE” initiative and what results they found.
Related Step
Evaluation
Age Group
13 to 16 years old
In order to evaluate the effectiveness of their “Sikker Trafik LIVE” (“Road Safety LIVE”) initiative (as introduced in more detail in example 4), the Danish Road Safety Council contracted the Danish Center for Social Science Research (VIVE) to conduct a randomised controlled trial (RCT).
For the evaluation of the “Road Safety LIVE” intervention, 93 schools were recruited in the summer of 2016. The evaluation used a waiting-list design, meaning that the schools were randomised to receive the intervention either as the experimental group in the autumn of 2016 or as the control group in the spring of 2017.
The evaluation furthermore used cluster randomisation, meaning that the randomisation was made at school level (as opposed to individual level). As schools would often book several “LIVE” visits at the same time (for different year groups), the randomisation at school level allowed the researchers to avoid situations in which pupils from the experimental group, who had participated in the “LIVE” visits, would influence those who had not yet participated (the control group).
The baseline measurement was conducted using an electronic questionnaire between 30 August and 16 September 2016, before the experimental group received the “LIVE” visit later that autumn. The post-activity measurement, also using an electronic questionnaire, was subsequently conducted in January 2017 – after the experimental group had received the “LIVE” visit, but before the control group would receive the “LIVE” visits later that spring. For both measurements, contact teachers were responsible for making sure that the pupils completed the questionnaires. 3,779 pupils responded to the baseline questionnaire, and 1,855 pupils responded in the post-activity measurement. (This decrease in participation is explained as there is less focus on the questionnaire after the intervention, and as teachers would often fail to understand that they had to complete the questionnaire twice.)
The questionnaire used during the measurements focused on knowledge, attitudes and behaviour, as these are the areas the Danish Road Safety Council seeks to influence with the “Road Safety LIVE” visits (see example 4 for the objectives set for the initiative).
The data acquired during the measurements was subsequently analysed, with the researchers applying two statistical methods to help account for any differences between the schools, and to ensure that their results were robust. The data showed that the experimental and the control groups were broadly similar.
With regard to knowledge, the evaluation results showed that a significantly higher number of pupils who had received the “LIVE” visit (the experimental group) knew which age group is most vulnerable in traffic, compared to those pupils who had not yet received such a visit. Significantly more pupils from the experimental group knew which three factors cause most road deaths and injuries, and therefore the pupils in the experimental group gained significantly more knowledge about road safety compared to the pupils in the control group. (Keep in mind that the control group had not yet had the LIVE visit when they filled in the questionnaire for the post-activity measurement. They would only have it later that spring.)
With regard to attitude, the evaluation results showed that during the post-activity measurement, significantly more pupils from the experimental group do not think it is acceptable to drive without a seatbelt, and they are less tolerant of risk behaviour such as texting while cycling and speeding in a car or on a moped. The results also showed significant differences between the regions in Denmark, for example significantly more pupils in the region around Copenhagen than in the other regions think that it is not OK to ride an illegally modified (‘tuned’) moped.
However, both baseline and post-activity measurements showed that while a considerable number of pupils (in both groups) stated that different forms of risk behaviour presented to them were not acceptable, approximately 20% of all pupils still think it is appropriate to drive 100 km/h on a 80 km/h road, 30% think it is acceptable to text while cycling, and 15% think it is acceptable to drive on an illegally modified (‘tuned’) moped.
With regard to the pupils’ behaviour, the evaluation results found that the pupils’ use of cycle helmets and using a mobile phone while cycling correlated with their parents’ behaviour, with both fathers and mothers who have had fewer years of education using their mobile phones significantly more when cycling. Also for other types of risk behaviour, the evaluation found that significantly fewer pupils in the experimental group whose parents had received less education objected when they were confronted with such risk behaviour.
Although more pupils from both groups said in the post-activity measurement that they listened to music while cycling, the increase was significantly smaller for the experimental group. In addition, pupils in the experimental group were better at objecting and asking others to change their behaviour than the control group. However, although these results were positive, they were not significant. Significantly more girls than boys in the experimental group objected to speeding.
The evaluation report concluded that positive impacts of the “Road Safety LIVE” initiative were found, particularly with regard to knowledge and attitude. Taking into account the size of the activity, the report concluded that its results are good, with the positive development in pupils’ knowledge and attitude possibly having a long-term impact. Furthermore, it suggested it would be beneficial to focus more on the young persons’ own behaviour in traffic, particularly when cycling.
Overall, the evaluation report concluded that the method in the “Road Safety LIVE” initiative of using ambassadors who tell pupils about their personal stories is effective.
More Information
LEARN! Manual Example #24: Randomised Controlled Trial of the “Road Safety LIVE” initiative