Friday, April 9, 2010

Web Based Training and Levels of Evaluation

Web Based Training has a variety of advantages. It can ensure consistency in training across an organization, since everyone will be subject to the same content. It can be much more cost effective, as it eliminates the need for travel and accommodation. It can be easy to evaluate, since built-in evaluation tools takes the manual work out of the equation.

But what are we talking about when we discuss evaluation?  I, like most of us was taught about Kirkpatrick's levels of evaluation. Don't know what that is?  Click here to learn more.

Level 1 is the easiest evaluation type to capture in web based training as almost all authoring tools include some evaluation tool that is capable of such. Keep them short to maintain as much accuracy as possible. If this type of evaluation is too time consuming, learners may arbitrarily check off all the same grading for each question thereby nullifying the results.

Level 2 evaluations are also not overly difficult to create in most authoring tools. Usually they are achieved much the same as the level 1 evaluation except that a grade can be assigned to the results. Several things to keep in mind are:
  • Be sure the evaluation addresses the learning objectives of the course.  Don't just ask questions for the sake of asking questions
  • If possible, use a randomize function to rearrange the order of the questions and the order in which the possible answers are displayed. this will make each version of your quiz different from the last
  • Choose a small number of questions from a larger pool of questions.  Again, creating a unique quiz each time the course is accessed
  • Make all possible answers appear plausible so the correct answer isn't easily guessed
  • Avoid True/False questions for the same reason as above
Remember that web based training can be difficult to evaluate when learning is in the affective (attitudes and feelings) or psychomotor (physical skill) domain.  Use simulations for systems training that can test the use of software without risk of making an error in a live system .

Now here is where things get tricky for evaluation of web based training. Level 3 can be one of the most costly types of evaluations for most organizations, especially those that have operations in multiple locations across the country.

Here is one suggestion. After a week or more has past, send an email to each student's supervisor. If your LMS can generate this automatically, all the better. Ask the supervisor to observe their employees for demonstration of the new skills in the work place. This assumes the supervisor has taken this training or is equally knowledgable about these new skills. Have the supervisor report back on the results.  Once the data is collected you can compair results and determine if the training has had an effect on the behavior of those that have completed it.  Hopefully the supervisors and the business as a whole are willing to buy in on this type of participation, otherwise you may have to perform this fieldwork yourself. 

Level 4 does require a commitment to follow up on your part, however the organization is likely already generating reports on results that effect the business. Some examples of these reports are revenue, costs, number of workplace accidents, etc. Make sure that you drill down to a level that matches the objective of the training.  This will ensure that some other area of the business isn't responsible for any changes and you can therefore show return on investment. 

In both level 3 and 4 you may want to also capture this data when you are designing your training. This way you will have a base line to compare your later results with, but you will also learn where the performance gaps truly are.

Tuesday, April 6, 2010

The G.R.O.W. Coaching Model

In one of my prior roles as an Instructional Designer it was required that my manager and I have a 30 minute "one on one" session scheduled. In many cases I would show up at my Manager's desk prepared to discuss my professional development, or hear some feedback, or discuss my current challenges. In most cases my Manager would ask if everything was okay. I would reply "Yup" and then I was invited to come by their desk next time when their might be something else we could discuss.

When there was something to discuss it usually went something like the coaching session you see in the following video:

You can see here the manager is making many of the common communication errors.  She's inattentive, focusing on her own work or issues, She is interrupting, and ansticipating what is going to be said next.  In general she is not providing any value to the situation.

I remember coaching sessions with other Managers that went extremely well. I never understood what process I was being guided through until I took a recent course on coaching.  This course got me thinking about some coaches that I have known.  I thought of those collegues from my past who were more guides than anything else.  They got me thinking about what I already knew in relation to the challenges I was facing.  By asking some key questions they unlocked the asnwers that were already within me.  Obviously I perceived them to be relatively passive in the coaching role but the results were far from that.

The coaching model we learned in class was the G.R.O.W. coaching model. This was a simple to learn four step method that would allow you as to really get at the potential of the person you were coaching.

The G in G.R.O.W. stands for goal. With every coaching session you need to identify what the person being coached wants to accomplish. Goals should always be specific, measurable, have a deadline. You can do this by simply asking questions like:
  • What are you trying to achieve?
  • Where would you like to improve?
  • What are you trying to do?
  • What is your goal?

It doesn't need to be complicated but you may need to help the person being coached to ensure their goals are as focused as they can be. Having the goal to lose weight someday isn't specific enough and certainly does not have a deadline. Goals like this often are never achieved for those very reasons.

The R in G.R.O.W. stands for reality. Having a goal is a great first step, however you need to establish your baseline. I can give you directions to drive to California, however if I don't know where you are starting from, those directions may be meaningless to you. You could end up driving in the completely wrong direction. Ask the person being coached questions like the following:
  • How are things going today?
  • Where do you find yourself at this time?
  • How far have you gotten?
  • What's the current situation?

The O in G.R.O.W. stands for options. The next step with the person being coached is to have them consider what their options are. If the person being coached is struggling with this you can simply present the options, allowing the person to make a choice on their own, or you can ask them more questions allowing them to unlock the possible choices on their own. Ask the person being coached questions like the following:
  • What do you think might be some options?
  • What choices do you think you have?

The W in G.R.O.W. stands for Willingness or what is the person being coached willing to do. Because this is their goal and not yours, it's important that this be calculated on their own. You can facilitate this by asking questions such as:
  • What is the best option for you?
  • What can you do now?
  • What will it take for you to achieve this goal?
I remember being taken through this process feeling frustrated at first when I was thinking:

"I wish they would just tell me the answer!"

Turns out that once a goal was realized I was quite pleased with myself that I was able to set a goal on my own, making the possibility of me achieving this goal so much greater. Coaching of course isn't teaching or training, however coaching works hand in hand with training. At some point your learners may come to you looking for answers you know they already have inside of them. You could just provide the answer, but think of the benefit of guiding someone through a little self discover using the G.R.O.W. model. Your learner is far more likely to take their personal goals more seriously if it comes from within.

Sunday, April 4, 2010

Multiple Intelligences

Back in 1983, Dr. Howard Gardner challenged traditional thinking that suggested people were either more intelligent or less intelligent.  Instead he suggested that there were different types of intelligences.  His theory is that each person contained eight different types of intelligences in varying degrees.  The eight intelligences are:
  1. Linguistic
  2. Logical-Mathematical
  3. Visual-Spatial
  4. Musical
  5. Interpersonal
  6. Intrapersonal
  7. Naturalist
  8. Bodily-Kinesthetic
Similarly to how you determine your learning style, you can consider your own experiences, tasks and activities and consider which areas you are most comfortable in.  It's important to note that this is not a type theory and again we each contain all eight intelligences in various degrees.  Traditional teaching may have focused in Linguistic or Logical-Mathematical.  Each of us has these intelligences, however if a person was primarily Intrapersonal and Musical, they may not excel in this particular example of education.

Here is an interview with Dr. Howard Gardner explaining Multiple Intelligences far better than I could:

One could incorporate this into their lesson design by having the students select from a list of activities that they can perform to complete that particular lesson.

Example: If the topic was the use of propaganda during World War II, students could select one of the following tasks:
  • Read an article on propaganda and explain it to the class
  • Create their own propaganda poster
  • Write a song that attempts to convince others of an idea
  • As a group shoot a video skit on propaganda
One thing you need to consider when using multiple intelligences in developing your lessons is in evaluation of the students.  Obviously something as simple as a multiple choice question is easy to evaluate, but in the example above evaluating the results could be very subjective and time consuming.  It's also very difficult to apply the same evaluation scale when each project is entirely different from one another.