LEARN! Manual Example #11
This example shows what considerations your pre-test design has to include.
Related Step
Pre-Testing
Age Group
n/a
When testing a new or existing activity (which has not previously been tested or which was tested a long time ago) you have to make a specific test design that matches the specific activity you want to test. The test design has to include considerations about at least the following:
What do you want to test?
Do you want to test how a specific activity is received and perceived by the pupils (do they learn what they should in terms of knowledge, reflection, attitude, intended behaviour, etc.)? And/or do you want to test how the teachers in charge of using the material receive and perceive the material?
Who is the target group? Who do you need to recruit for testing?
Is the target group for instance specific age groups/classes from different regions in your country, or classes from the regions/areas where most pupils are killed or injured in traffic? Or is it a combination of schools that teach specific age groups/classes in traffic safety and schools that do not? Or is the target group composed of teachers who teach a certain subject, or instead teachers who would use a new teacher´s guide for a certain activity?
And, when you have defined who to recruit for testing, how will you recruit them? Will you do this by calling different schools? Or via a newsletter to schools and/or teachers? Or via social media, etc?
What methods will you use?
Will you use qualitative methods – for instance (participant) observation, interviews, and/or focus groups? Or quantitative methods – for instance surveys of pupils and teachers? Or will you use a mixture of qualitative and quantitative methods? And when will you do the interview or survey? Before a teaching setting, or after, or both?
What do you want to know?
Presumably you want to test if the activity is used as intended. That is, do the pupils learn what they should according to national goals, your theory of change one-pager, and what is intended with the activity in terms of learning, knowledge, attitude, competences, etc? It is relevant to use the 5E model here to see if and how the activity is being taught and perceived in terms of engaging, exploring, explaining, elaborating and evaluating.
What resources do you have?
In terms of finances and/or personnel to conduct the test. Do you have a short or longer time schedule? And will you provide remuneration (gift cards, etc) for the participants?
Who will do the testing?
Will it be an external party (e.g. consulting firm, research institute, etc), an in-house documentation unit (who are not in charge of the development of the activity, but only of testing, monitoring and evaluating it), or will the testing be conducted by the developers of the activity or colleagues of them?
While the first two options are professionals and neutral, one should be aware that the latter option is not neutral and the people running the tests are unlikely to have the training to conduct testing. It is therefore recommended to ask the external party or the in-house documentation unit to conduct the testing.
However, if this is not possible (for instance due to limited resources) it is much better that the developers of the activity do the testing than not have testing at all – as long as they aim to stay neutral and do not get discouraged by criticism of “their baby”.