Cheaters

One problem that e-Learning designers face is designing with cheaters in mind. Early in my career as an Instructional Designer we used e-Learning software that presented evaluations on a single page in whatever order you entered it. Once an evaluation was submitted, learners saw a results page that revealed all the correct answers. We began to see a trend where the first learner from each location across the country would receive a lower average score, while each subsequent learner would score much higher. What we suspected turned out to be true. The first learner was sharing their results with other learners, resulting in much higher scores for the remaining learners.

To resolve this we switched to a new authoring tool capable of providing different evaluations to each learner who took our online courses.

The key features of this authorware's evaluations were:

1. Questions in Random Order - The authorware was capable of randomizing questions so that the order of the questions was never the same
2. Randomly Displayed Answers - while some of the questions may get repeated, the answers themselves appeared in random order. In otherwords, what might have been answer a) the first time the course was run, may become answer c) the next time
3. Pulling from a Larger Pool of Questions - the authorware was able to generate its random set of questions from a larger pool of questions. For example, we might have 10 questions presented from a possible list of 30 questions.  This would also present a completely new set of question each time the course is run

Employing these techniques certainly reduced the number of cheaters who were taking our online courses, however improving the quality of questions helped as well.  We almost entirely stopped using True/False questions.  The reason is that a True/False question tends to lead the learners to the correct answer.  In an organization where keeping employees positive about the products and services they sell, the answer to a True/False question is usually true.  You don't want to draw attention to what isn't true about a product or service.

Also ensuring that all the answers in a multiple choice question are at least plausible is important.  If the learner can deduce the correct answer by eliminating all the improbable answers, then you really haven't evaluated their knowledge of the topic.  All you have tested is the learner's skills of deduction.

Another thing relating to the plausibility of possible answers is the length of the answers.  Back in high school, I learned that when I didn’t know the correct answer to a multiple choice question, the longest answer was often the correct answer.  The reason for this is that designer of an evaluation may have to be careful about phrasing correct answers to ensure there is no room for interpretation.  The same is usually not true of wrong answers.  Obviously a wrong answer only needs precise wording when there is risk of it being a correct answer.

Ferris Bueller's Style of Training

Many of my generation think back fondly to the film Ferris Bueller's Day Off.  It is a story of the popular yet rebelious teenager who skips school to go on an adventure in the city of Chicago with his best friend and his girlfriend.  Throughout the film, the school principle is one step behind them trying to catch them in the act of ditching school.  Of course it almost goes without saying that Ferris, like all kids from the 80s, is much smarter than the principle of the school and manages to avoid Mr. Rooney with ease.

When I reflect on the film now as an adult and professional within the adult training field, I actually see an interesting thing occur in the film.  Ferris and his two friends actually receive a great education by skipping school, and there is an interesting sub message within the film.  In fact, Ferris and his two friends could easily have experienced many of the things on their day off during a class field trip.  I'm sure they experienced and learned more than their fellow students who were back in school listening to Ben Stein's lecture about the great depression.

During their trip they visited the Sears Tower and learned about it's enormous height and how it was built, Saw the Chicago Mercantile Exchange and witnessed how commodities are traded, They visited the Art Institute of Chicago and saw many famous works of art.  I dare say they learned a great deal about themselves as well.  They explored their own feelings about themselves and one another and what the future has in store for them.  The Irony is that their principle spends all this time trying to catch up with them so he could ultimately return them to the environment of books and classrooms where they likely wouldn't have learned all that much.

As teachers and trainers we should look to Ferris Bueller and get out of the text books and classrooms and get our students to truly experience life.  They will learn so much more than just what they read in books and lesson guides.

Motivation

A couple of years ago I went to Chichén Itzá in Mexico. I was amazed that the ancient Mayans built structures like the pyramid of Kukulkan seen here using only their hands and the simplest of tools. Even more amazing is the quarry for the stone used was over a mile away and the Mayans had not yet invented the wheel.  What would it have taken to motivate a group of people to build such an object?

I'm always amazed with peoples ability to stay motivated to complete a job. As someone involved in training I'm very interested in motivation and I'm always looking for new ways to tap into this ability.  Dictionary.com uses words like inducement or incentive when describing motivation.  I think of motivation as the 'reason' behind anything we do.

When I was a sales manager for a large chain of retail stores, what I found interesting was that money wasn't always a motivator for everyone.  This surprised me because the people in question had all chosen a career of sales.  Certainly sales is something that involves money.  I had several employees that were motivated by food and/or drink.  A plate of chicken wings and a pitcher of beer was usually all it took, however if I offered the same in cash, these individuals were not as motivated as the food and drink equivalent.


Within training, motivation comes from the affective domain.  This is the emotional side of training that involves how someone feels about a topic or task.  This is probably one of the most difficult things to train as peoples feelings vary from one to another.  You need to have people unlock there own potential much like a coach of a sports team.  Simply telling them what to do will not provide the motivation on it's own.  Instead have your learners figure out what the results of the change will mean to them and what types of rewards those changes will bring.  Like most things that we as professional trainers teach,  the most engaging stuff will be the things the learners have to do for themselves.

Blooms Taxonomy: Part Three

The affective domain is the third domain of learning included in Bloom's Taxonomy.  Skills in this domain tap into the way people feel about the learning material.  The goal of this type of learning is usually to get people to react emotionally and get in touch with attitudes toward the material.

Training within the affective domain is probably the hardest domain to achieve, however when it is done well, it literally can be a call to action or change the emotions of an entire group.  I have posted this video before, but here is an example of affective training.  I believe the goal that Dr. Randy Pausch had when giving this lecture was to remind us that how we live our lives is important and we should not waste the precious gift that life is.  I think you will agree that he is very capable of tapping into our emotions and perhaps changing how we think or feel about death.



Many organizations use this type of training to change the attitudes of their employees.  Perhaps they have had a fundamental shift in ideals that they wish to convey to their employees, or perhaps they are attempting to correct behaviors or attitudes they have become aware of.

Like the other domains there are levels of learning within the affective domain.  They are:

1. Receiving - learners passively pays attention through listening and/or observing.
2. Responding - learners actively participates by reacting in some way demonstrating their willingness to be involved.
3. Valuing - learners agree with the material that is presented not just through compliance but because they have a belief in the content itself.
4. Organizing - learners put together values and compare, relate and elaborate on the material that is presented.
5. Characterizing - the learner has adapted these values so as to become apart of their makeup as a person.  These values will affect how the learner behaves and are characteristic of who they are.

When stating your objectives and describing your learning activities will require action verbs that are appropriate for this type of learning.  Below are some examples:

Receiving – ask, attend, choose, describe, give, identify, listen, receive, use, view, watch

Responding – communicate, comply, consent, conform, contribute, cooperate, discuss, follow-up, greet, help, inquire, participate

Valuing – accept, adopt, approve, complete, choose, commit, desire, differentiate, display, endorse, explain, express, form, initiate, invite, join, justify, propose

Organizing – adapt, adhere, alter, categorize, classify, combine, compare, complete, defend, explain, establish, formulate

Characterizing – advocate, characterize, defend, devote, disclose, discriminate, display, encourage, endure, exemplify, function, incorporate, influence, justify

By the way, Dr. Pausch did eventually pass away and while he was still with us, he managed to fulfill another dream of his.  He was invited to play a character in the latest Star Trek film.  He is on the bridge of the USS Kelvin in the opening scene of the film.

Blooms Taxonomy: Part Two

The second domain from Bloom's Taxonomy that I am going to discuss is the psychomotor domain.  The psychomotor domain is where the learning achieved through the cognitive domain is demonstrated with physical skills.  This is an important domain when you are training skills that would be considered manual labour.  This domain can be broken down into 5 levels as follows:

1. Imitation - learners will attempt to duplicate an act that has been demonstrated to them.  Error is expected and encouraged as a learning opportunity for the learners.
   
2. Manipulation - Continual practice of the act so that it becomes habitual and confidence is built.
   
3. Precision - The skill has been attained.  At this stage refinement or further accuracy become the goal.
4. Articulation - At this stage learners will adapt their movement patterns to accommodate special needs or to meet a specific problem.
   
5. Naturalization - Naturalization occurs when the skill becomes an automatic response.

In this domain's case, learners will likely start out with the first level; imitation and then progress to manipulation, precision and so on.  In the psychomotor domain you would write your learning objectives and learning activity instructions based on the current level of performance that your learners are currently able to demonstrate.  For example you may have beginner, intermediate, and advanced courses that gradually move the learners from Imitation right up to naturalization.  You would most likely want each course to be a prerequisite of one another to ensure that the appropriate skills needed are achieved first.

An example of training in the psychomotor domain would be physical side of training for a pilot.  In the beginning you would demonstrate the skills for them to imitate.  Likely this would be done in a simulator for safety reasons.  The potential pilot would be encouraged to make mistakes and see what the results are.  Continual practice would occur during the manipulation stage so that confidence is built up and actions become habitual.  At this stage the potential pilot would like gain experience in real air craft.  Further practice would be used to gain the level of precision.  Articulation would teach them to be adaptable when needed, perhaps how to handle emergency situations.  In the final stages of their training, naturalization would occur.  The pilot would simply react appropriately during different situations without having to spend time thinking about each situation.

The types of action verbs you would use for stating objectives and writing learning activities would be:

Imitation - begin, assemble, attempt, duplicate, follow, mimic, practice, reproduce, try

Manipulation - acquire, complete, conduct, do, execute, improve, maintain, make, manipulate, operate, pace, perform, produce, progress, use

Precision - accomplish, advance, exceed, master, reach, refine, succeed, surpass, transcend

Articulation - adapt, alter, change, excel, rearrange, reorganize, revise, surpass, transcend

Naturalization - arrange, combine, compose, create, design, refine, originate, transcend

Like before many of these words are interchangeable and certainly could apply in more than one level (especially during imitation, manipulation, and precision).  Remember you are not limited to these words, there are likely thousands of action verbs you could use for every level of Bloom's Taxonomy.

Blooms Taxonomy: Part One

Some Instructional Designers moan about going through the exercise of using Blooms Taxonomy while designing training.  I submit that when you have attended training sessions that over promised yet under delivered, it’s likely because the designer of the training ignored Bloom's Taxonomy.  The most important aspect of Bloom's Taxonomy from the perspective of your learners is that they will receive exactly what was promised during any type of learning activity.

If I promised you the ability to rebuild an automobile engine by the end of my course and simply stood at the front of the class and forced you to memorize facts and figures about cars, you could no more rebuild an engine than you could before you started.  Essentially it’s like bad advertising.  Writing proper learning outcomes will also force you to further develop your learning activities so they do achieve the goals of training.

Bloom's Taxonomy can be broken down into the three domains of learning: cognitive, psychomotor, and the affective domains. Most training and education falls within the cognitive domain so in this first of three parts on Blooms Taxonomy that I will discuss.  The cognitive domain can be broken down into 6 levels of learning.  Here they are explained using some interesting scenes from Pirates of the Caribbean (just for fun).




Cognitive Domain

1. Knowledge is simply the ability to remember fact or content. 
2. Comprehension is the ability to understand the material through interpretation, translation or the ability to predict consequences of the material.
3. Application is the ability to independently apply material in new situations.
4.  Analysis is the ability to break down material into it's individual parts.
5. Synthesis is the ability to create something new from the material.
6. Evaluation is the ability to judge the value of the material

So how does this apply to planning your lessons?  Simple, your goal should be to achieve the higher levels of the cognitive domain.  While knowledge and comprehension are important, memorization and understanding is no substitute for being able to do or create something new or to judge it's value.  This is where true learning occurs.

Secondly to ensure that you direct your learners toward these higher levels, you need to choose the correct language in both your learning objectives and in your instructions for learning activities.  Selecting the correct action verbs will help your learners prepare for what they must do.  Below are some examples of words that will tell your learners what's expected of them and which level of learning these types of activities accomplish:

Knowledge - select, identify, label, match, recall, recite, reproduce

Comprehension - describe, define, explain, illustrate, restate, rewrite


Application - demonstrate, apply, develop, organize, operate, produce, solve, modify

Analysis - compare, distinguish, analyze, breakdown, classify, separate, subdivide

Synthesis - combine, compile, compose, conceive, construct, create, design, generate, invent

Evaluate - appraise, conclude, contrast, criticize, decide, defend, discriminate, write (a review)

Of course several of these action verbs can fall into more than one level of learning, and there are many more possibilities as well.  This should allow you to take your learning activites and determine what level of Blooms Taxonomy they achieve.  For example if I had a learning activity that stated:   

"Please select the correct answer from the following list of wrong answers." 

...you would know that this only achieves the knowledge level.  Simple memorization is all that is required to answer this question.  It doesn't determine if the learner actually comprehends the knowledge.  Of course that might be fine.  Perhaps it isn't required that the learner know why something is correct, just that it is correct.  If a learner's job requires the ability to analyze a situation and act appropriately with that information, you are going to want to design course objectives and learning activities that tests their ability to analyze, synthesize, and evaluate the material you provide them.  More difficult to do? Yes.  More rewarding for your learners? Absolutely.

Training for Different Language and Culture

Many instructional designers make the mistake of only thinking in their primary language, and forgetting that some things are simply not translatable. I try to keep simplicity in my course design so as to avoid the additional challenge that multiple languages creates. Designing only with English in mind is clearly a mistake when someone lives and works in a country like Canada. Here in Canada we have two official languages, English and French. Geographical speaking the majority of Canada speaks English, while French is mostly spoken in the province of Quebec.

A lesson designed for a new hire course revolved around the famous “Who is On First” routine by Bud Abbott and Lou Costello. This lesson illustrated how easy it is to misinterpret what someone is saying. Using tasteful humour such as this example can be very effective in training, however because this comedy routine was never translated nor performed in any other language except for English, it could not be used in French speaking Canada. To do so would create more initial work for the designer, and creating a break in continuity between versions. This would also create additional work in maintaining future updates as well.



In cases where lesson material is designed only for learners whose first language is English, many learners who first language is something other than English will often fall behind the rest of the class. The reasons for this are many; however some examples might be they:

• Only have a basic understanding of the language
• Do not share a cultural background and will miss certain references
• Are not familiar with technical terms or jargon

Like as in language, you should be very careful with humour when it comes to cultural differences. A humorous video may seem harmless, yet it may completely go against the grain of another culture. The resulting offense to another culture can detract from the purpose of training and force you to spend time putting out unnecessary fires.

The following things to avoid in your training may seem obvious; however you would be surprised how often people break with these rules. Avoid opinion or discussion that includes...

• Profanity of any kind
• Religion
• Politics
• Cultural stereotypes
• Sexuality
• Gender
• Sexual orientation
• Ethnic or racists remarks

There are several things you can do to ensure you don’t cross any lines. If you are unsure and think that a piece of material may be questionable, play it safe and simply leave it out. In this case it’s far better to avoid a potential problem with language and culture then to try and recover from it later.

Secondly you should always conduct a pilot or your training with a safe group of your peers. Try to include a culturally diverse group or bilingual group when possible, they certainly can point out problems that you may miss or may simply not be aware of.

Teaching Like Oprah

Although some lectures can be very exciting, you may be afraid to bore your learners to sleep.  Here is a learning activity that may prove to be more engaging than the traditional lecture. 

Ask for a volunteer to play the part of a talk show host.  You will provide them a list of questions that must be asked of you the instructor so that the topic is well covered.  Remember you are the one being intereviewed.  In addition to the scripted questions, invite the host to ask their own follow up questions as they see fit.  Toward the end of the activity, the talk show host will call upon questions from the audience to also address any knowledge that was missed.  Use this time to evaluate what the students learned and address any missing pieces of knowledge.

The advantage of this learning activity is that it takes the learners from the more passive role of listening to a lecture to an active part of a mock television show.  It may prove to be more engaging and will take some of the pressure off all involved as this will create an informal atmosphere.

Take a look at some popular talk shows and try to emulate some of the segments that they include to get the audience more active in participating with the show.  You may want to avoid shows like Letterman and Leno and focus more on the self help style of shows like Oprah, and Dr. Oz.  If you are successful, I promise your lesson will be far more memorable than a boring lecture.

Learning Strategies for Teaching Adults with Learning Disabilities

We have all heard the term learning disability and may know one or two learning disabilities themselves, but what is a learning disability?  Simply stated, a learning disability is a disorder that affects the acquisition, retention, understanding, organization or use of verbal and/or non-verbal information.  This entry discusses those who have average to high intelligence, yet have an impediment that prevents them from learning skills or knowledge in the same method or rate of speed as others with similar levels of intelligence.

Many learning disabilities are genetic or possibly caused by an injury. These disorders can include difficulties in the following skills:

• Reading
• Writing
• Speaking
• Visual recognition
• Math
• Spatial or Mechanical
• Social

Learning disabilities are more common than most people expect.  As trainers of adults, there is a likelihood that about 20 percent of our learners will have some form of learning disability.  Many learners who are approximately 35 or older may never have been tested for learning disabilities, and while some of your learners will be aware of their challenges, others may wrongly believe they are just not as smart as others.  For those that know they have a learning disability, it is up to them to inform you of their additional needs if they so choose.  In Canada having access to accommodation for learning disabilities is considered a human right.  Denying a learner the accommodation they may require can get you into legal trouble.

Some people argue that accommodating those with a learning disability gives an unfair advantage over others.  For example, you may provide a learner who has difficulty reading, more time to complete a written test.  In this example, the additional time is needed to comprehend the question as it is written.  The additional time you provide the learner does not give them necessarily more time to answer the questions.  I like to compare these types of accommodations to other tools we use to assist our lives.  For example it is common to wear eye glasses when you have trouble seeing or reading, yet no one would argue that wearing glasses gives those an unfair advantage.

Of course your learners with learning disabilities will still need to accomplish the learning objectives of the training.  If the model of performance is to type 40 words per minute, then the student will still need to demonstrate this by the end of the training.  Providing accommodation does not give them an unfair advantage, but rather removes or reduces any disadvantage their disability presents.  Here are some examples of accommodation you could provide in the event that someone with a learning disability requires it:

• Course outlines to reduce the need to take notes in class
• Copies of materials such as overheads, diagrams, PowerPoint files, etc.
• Access to alternative testing methods (oral, or online)
• Additional clarification of questions on tests
• Use of a calculator during math problems
• Extended time to complete evaluations (usually 1.5x)
• Open book testing
• Extra tutoring

Because so many learners with these types of disabilities will not be identified, there are some things you can do to ensure you are as accommodating as possible.  Use...

• Easy to understand agendas so learners will know what to expect throughout the course
• Clear instructions both written and explained verbally
• Key concepts or terminology at the beginning of each lesson
• Clear visuals in your printed and overhead material (images should clearly look like what they are depicting)
• Physical examples such as models or actual items
• Point form to increase comprehension
• step-by-step instructions to break down tasks
• Group work where learners get to choose specific or unique roles within the group
• Words, pictures and sound to convey ideas (offering choices in how to learn)
• Multimedia when possible rather than reading or lecturing
• Job aids that can be referred to upon returning to the workplace

Here is a video that demonstrates how difficult it can be for someone who has a learning disability related to reading:




Similarily here is another video that deals with how difficult a learning dissability can affect visual information:

Unfortunately much of the resources available online are geared for children rather than adults, however many of the accommodations can apply to both children and adults equally.

Learning Disabilities Association of Canada http://www.ldac-taac.ca/
Learning Disabilities Association of Ontario http://www.ldao.ca/
Learning Disabilities Association of America http://www.ldanatl.org/

A Better Level One Evaluation

For those that are new to training, level one evaluations refer to Donald Kirkpatrick's methods or levels for evaluation. I've discussed this before, however a level one evaluation measures the learner's reaction to training. Simply put; did they like the training or not.  We've all has these handed out to us moments before we dash out of a training session. These are the evaluations you fill out where you rate your feelings about the training and the instructor. Sometimes we refer to these as smile sheets.

I believe that very little thought is put into filling these out by most of your learners.  The majority of your learners will very quickly determine which side of the check boxes are either good or bad and then arbitrarily check off all the answers to reflect their overall feeling. Usually the learners don't take time to read each question and select an answer that is truthful or accurate about how they feel.  This is why I feel there may be a better way to administer a level one evaluation.

Here is an alternative to this method. Rather than waiting until the end of a course or session, pick a half way point to administer an evaluation. If you are running a one day session, hand it out just after lunch. If you are running a two day session, hand it out at the beginning of day two. At least this way they will take the appropriate amount of time to complete this activity, rather than rush through, hoping to get the extra time for a break or their departure home.

I recommend that you change the questions to reflect that you are only part way through the session. In addition avoid a rating scale. Ask open ended questions so that you can pinpoint exactly what concerns your learners have. Ask questions like:

• What am I as an instructor doing well?

• What would you like to see more of today?

• What would you like to see less of today?

You could also review these with the class and build an agenda to ensure that the topics identified in this evaluation are covered during the rest of the session. You may want to remind your learners beforehand that while you are open to covering additional topics you will need to stay on topic to achieve the learning objectives of the course.  If you are teaching your learners about a new point of sale system,  having an open discussion about payroll will not achieve the goals of today's training.

The advantage of this style of evaluation is that you can shift gears in your teaching style and hopefully provide what your learners need to be successful.  Next time you have a course, try this as either a simple alternative to what you have done in the past or simply add it to your existing agenda.  I'm sure you will see positive results.