Evaluating Training

Evaluating training is crucial for any organization. Without it, we cannot show if training is a worthwhile activity. The most widely used and accepted standard to evaluate training is to use Professor Donald Kirkpatrick’s levels of learning evaluation. The theory is that the more of these you use, the more complete your evaluation of training will be. Also achieving evaluation on the higher levels, while difficult, is a more effective measure of training success than just the lower levels which are easier to achieve. These levels are as follows:

Level 1 – Reaction
How did the learner feel about the training?
Level 2 – Learning
Did the learner retain the knowledge or skills taught?
Level 3 - Behaviour
Did the learner apply the new skills or knowledge in their daily life?
Level 4 – Results
Did the skills or knowledge have an impact on the organization’s results?

Level one is probably the easiest learning evaluation to achieve. In the training industry, we sometimes refer to these as smile sheets. These are typically handed out at the end of a session or coded at the end of an e-learning course to simply capture the feelings that the learner had with the training itself. It’s a well-known fact that many learners arbitrarily fill these out to either exit the class a few minutes earlier, or simply rush through them because they don’t really care about the data being collected. I always learned to take these with a grain of salt since I conducted training within an organization where my learners were mandated to participate.

One suggestion to make these more effective is to introduce an earlier level one evaluation to gauge what the training is like for the students so far. Perhaps if you are teaching a one-day session, hand out a modified feedback form after the first morning break. If it’s a multiple-day session, hand out the form at the beginning of day two and so on. You could ask your learners what they would like more of, what they would like less of, and what they would like to see that currently has not been addressed so far. You must be careful to preface this by suggesting to your learners that you may not be able to accommodate all suggestions as you must stick to the course objective. The reason you do this is to maintain control and not disappoint those who suggest something way out in the left field. Being able to change your teaching style or accommodate a particular learning style is hugely beneficial. It’s also beneficial to give your students time to complete feedback when there are no distractions such as getting through rush hour traffic a few minutes earlier.

A level two evaluation is a little more challenging than level one, however, most trainers include some type of test or quiz upon the completion of training. This should determine if the learning objective has been achieved. Most organizations use a combination of fill in the blanks, multiple-choice, and true-false type questions. Whenever possible a practical evaluation is preferred. For example; if you were teaching learners how to fill out a tax form, the level two evaluation should have them fill out the very same tax form. Answering some additional questions to address variations on how or why a form may be filled out a different way could also be beneficial, but training should concern itself more with doing as opposed to knowing or understanding.

Here is where evaluation gets more difficult. Level three often requires fieldwork and can be very labour intensive. If you trained employees all across the country to perform a certain task on the job, you need someone in all locations to observe and report back to the organization whether or not the task is being performed to a satisfactory level. Typically each location has a Manager who can perform such evaluations; however, there might be a margin of error created by Managers who do not align themselves with the organization’s objectives. When completed properly, a level 3 evaluation can teach you that there may be a question of adoption. The results can either be positive or negative, however negative doesn’t always mean that training was to blame for poor results. There may be a bigger issue related to the commitment that the organization needs to address.

Level four is also a challenge for many organizations. In all likelihood, the information mechanism for gathering data for a level four evaluation probably already exists. Proving that any lift in results is at least partially from training can be difficult to show. Of course, other areas of the business tend to want to snap up whatever credit for success there is for them. If a business launched a new product, often the marketing department will take credit for the improved results rather than allowing training to enjoy or share the success. Probably the only way to ensure that you attribute training success to the training is to ensure there is a test group or a control group. For example, if one group of employees took the training while another group’s training is delayed you could compare the results of the two groups to show how effective the training was for those who benefited by it.

Hopefully, this entry gave you some insight into how you can improve your organization’s methods to the evaluation of training. It really is beneficial to show the return on investment in training back to the organization and not just allow the smile sheets to be the only argument for why training is necessary.