Web Based Training and Levels of Evaluation
Web Based Training has a variety of advantages. It can ensure consistency in training across an organization, since everyone will be subject to the same content. It can be much more cost effective, as it eliminates the need for travel and accommodation. It can be easy to evaluate, since built-in evaluation tools takes the manual work out of the equation.
But what are we talking about when we discuss evaluation? I, like most of us was taught about Kirkpatrick's levels of evaluation. Don't know what that is? Click here to learn more.
Level 1 is the easiest evaluation type to capture in web based training as almost all authoring tools include some evaluation tool that is capable of such. Keep them short to maintain as much accuracy as possible. If this type of evaluation is too time consuming, learners may arbitrarily check off all the same grading for each question thereby nullifying the results.
Level 2 evaluations are also not overly difficult to create in most authoring tools. Usually they are achieved much the same as the level 1 evaluation except that a grade can be assigned to the results. Several things to keep in mind are:
Now here is where things get tricky for evaluation of web based training. Level 3 can be one of the most costly types of evaluations for most organizations, especially those that have operations in multiple locations across the country.
Here is one suggestion. After a week or more has past, send an email to each student's supervisor. If your LMS can generate this automatically, all the better. Ask the supervisor to observe their employees for demonstration of the new skills in the work place. This assumes the supervisor has taken this training or is equally knowledgable about these new skills. Have the supervisor report back on the results. Once the data is collected you can compair results and determine if the training has had an effect on the behavior of those that have completed it. Hopefully the supervisors and the business as a whole are willing to buy in on this type of participation, otherwise you may have to perform this fieldwork yourself.
Level 4 does require a commitment to follow up on your part, however the organization is likely already generating reports on results that effect the business. Some examples of these reports are revenue, costs, number of workplace accidents, etc. Make sure that you drill down to a level that matches the objective of the training. This will ensure that some other area of the business isn't responsible for any changes and you can therefore show return on investment.
In both level 3 and 4 you may want to also capture this data when you are designing your training. This way you will have a base line to compare your later results with, but you will also learn where the performance gaps truly are.
But what are we talking about when we discuss evaluation? I, like most of us was taught about Kirkpatrick's levels of evaluation. Don't know what that is? Click here to learn more.
Level 1 is the easiest evaluation type to capture in web based training as almost all authoring tools include some evaluation tool that is capable of such. Keep them short to maintain as much accuracy as possible. If this type of evaluation is too time consuming, learners may arbitrarily check off all the same grading for each question thereby nullifying the results.
Level 2 evaluations are also not overly difficult to create in most authoring tools. Usually they are achieved much the same as the level 1 evaluation except that a grade can be assigned to the results. Several things to keep in mind are:
- Be sure the evaluation addresses the learning objectives of the course. Don't just ask questions for the sake of asking questions
- If possible, use a randomize function to rearrange the order of the questions and the order in which the possible answers are displayed. this will make each version of your quiz different from the last
- Choose a small number of questions from a larger pool of questions. Again, creating a unique quiz each time the course is accessed
- Make all possible answers appear plausible so the correct answer isn't easily guessed
- Avoid True/False questions for the same reason as above
Now here is where things get tricky for evaluation of web based training. Level 3 can be one of the most costly types of evaluations for most organizations, especially those that have operations in multiple locations across the country.
Here is one suggestion. After a week or more has past, send an email to each student's supervisor. If your LMS can generate this automatically, all the better. Ask the supervisor to observe their employees for demonstration of the new skills in the work place. This assumes the supervisor has taken this training or is equally knowledgable about these new skills. Have the supervisor report back on the results. Once the data is collected you can compair results and determine if the training has had an effect on the behavior of those that have completed it. Hopefully the supervisors and the business as a whole are willing to buy in on this type of participation, otherwise you may have to perform this fieldwork yourself.
Level 4 does require a commitment to follow up on your part, however the organization is likely already generating reports on results that effect the business. Some examples of these reports are revenue, costs, number of workplace accidents, etc. Make sure that you drill down to a level that matches the objective of the training. This will ensure that some other area of the business isn't responsible for any changes and you can therefore show return on investment.
In both level 3 and 4 you may want to also capture this data when you are designing your training. This way you will have a base line to compare your later results with, but you will also learn where the performance gaps truly are.
