3 tips for anyone to design or evaluate training

Today, I’ll be telling you about my brilliance. I am SME, hear me roar!

We have no choice but act in constant learning mode, from lunch and learns to new policy roll out to new software to train-the-trainer to skills development.

Forget those who embrace training and see themselves as lifelong learners.

As the saying goes “if you’re not getting better, you’re getting worse” and organizations spend on training:  in 2011, United States organizations spent $156.2 billion on training, ~3.2% of payroll.

To CEOs the 2 most important learning metrics map to:

1. Business strategy impact
2. Return on Investment (ROI)

For simplicity sake let’s bundle training, education, and learning. I can effectively split hairs in another post, but here let’s keep the three together.

The biggest challenge to a training ROI and how people and organizations embrace, accept, or reject training is constant training attempts by people who know little about effective communication and less about what makes effective training.

Let’s address the elephant in the room: the Subject Matter Expert (SME):

1. An SME does not correlate to effective trainer
2. An SME attempt to train may do more harm than good

So, we spend a lot on training, before we even factor in people’s time, but as the American Society for Training & Development loves to say, Telling Ain’t Training.

Effective Communication is Effective Training

In my organization development role I advance a very distinct point: communication is training and training is communication. The goal of either: convey information to cause reaction.

Training as communication simplifies the change management game.

While many change management efforts count on communication, the majority I review leave training out. Worse, the past 15 years, most projects I work on at least scope some form of communication before project launch, but I can count on one hand the amount of projects that scope training before project launch.

Also, training as communication holds the subject matter expert accountable for effective communication and how and if they are the best source or should deliver communication.

Just because you know a topic lends no ability that you communicate effectively. Depth of skill does not correlate to ability to train.

Finally, good communication is measurable, good training is measurable.

Evaluate Training

Outsourced training is a perceived solution that many organizations take. The objective expectation to outsource is an increased return with training subject matter experts, either as a product vendor or a specialized training partner.

The trouble I continue to run into:  groups who offer training, themselves, have with a quality review of the list below, reveal little evidence as qualified or capable trainers.

Here are three tips to evaluate training quality. Use this list to also provide quality assurance to help design training. Effective training needs to cover three key areas and how the tips presented:

1. Communication Quality
2. Training Quality
3. ROI Quality

Tip 1: Communication Quality

Here is a communication quality check list to evaluate how to communicate. Seek these in design and in evaluation:

1. Know your audience — clearly the easiest maxim for effective communication. What does this mean? There are multiple personas with multiple needs, identify what their needs are. Does your training answer WIIFM? (What’s In It For Me?). You design for personas, for example: Enterprise software rollout communication to meet an executive assistant’s needs different than a sales representative’s need.
2. Learning goals — Identify learning goals of the session. You might find a need for multiple sessions. And since you have multiple training community personas the multiple goals relate to how they work and what they need to get their work done [more insight below within Adult Learning Styles].
3. Agenda — Review the agenda, what are the topics to cover or leave
4. Tell them what you will say; say it; tell them what you said – An old communications maxim as old as Hammurabbi’s code. Prepare your audience for a topic, communicate about the topic, then recap what (three take-aways; see point 5, below) you told them.
5. Three, is a magic number — McKinsey Consulting recommends no more than three points in a PowerPoint slide; Schoolhouse Rock! says three is a Magic Number, in music there is a reason they call a three-piece band a power trio. Keep your points, objectives, goals to three and keep things in digestible chunks.

Tip 2: Training Quality

This is a list to gauge if training is set for real results and about learning style. Adult learning styles require a set of diverse needs to meet how they learn best.

About 10 years ago I found this 1-page list 30 Things We Know For Sure About Adult Learning from Honolulu Community College, originally written in 1984, by Ron and Susan Zemke.

This list has been a go-to source for me to review the myriad of conditions I want to account for when I design or how I assure design quality. The Zemke’s brilliantly separate the 30 items into (guess what!) three categories. Here are samples from each section:

1. Motivation to Learn
   • The learning experiences adults seek out on their own are directly related – at least in their perception – to the life-change events that triggered the seeking.
   • Adults are generally willing to engage in learning experiences before, after, or even during the actual life change event. Once convinced that the change is a certainty, adults will engage in any learning that promises to help them cope with the transition.
2. Curriculum Design
   • Program designs need to accept viewpoints from people in different life stages and with different value “sets.”
   • A concept “anchor” or explanation from more than one value set appeals to more than one developmental life stage.
   • Adults need integrate new ideas with what they already know if they are going to keep – and use – the new information.
   • Information that has little “conceptual overlap” with a context to what is known or acquired proves more difficult to digest.
   • Fast-paced, complex, or unusual learning tasks interfere with learning concepts or data intended to teach or illustrate.
3. In the Classroom
   • Adults bring a great deal of life experience into the classroom, an invaluable asset to be acknowledged, tapped and used. Adults can learn well – and much – from dialogue with respected peers.
   • Instructors who have a tendency to hold forth rather than facilitate can hold that tendency in check–or compensate for it–by concentrating on the use of open-ended questions to draw out relevant student knowledge and experience.
   • Adults have something real to lose in a classroom situation. Self-esteem and ego are on the line when they are asked to risk trying a new behavior in front of peers and cohorts. Bad experiences in traditional education, feelings about authority or preoccupation with events outside the classroom affect in-class experience.

You will quickly value this collection on adult learning insight. Print the list, laminate the list, save the list on your computer, tattoo 15 items on the inside of each eyelid; 2 x 15 = 30, see there is that “3 (three)” again.

Tip 3: Measure ROI Quality

Here, ROI measures both organization Return on Investment and Return on Involvement.

You can not change what you can not measure. The value of training is on trial, a trial to prove training’s return on investment. With money scarce you need to identify the dollar impact on training. Here is your greatest ally: The Kirkpatrick Four-Level Evaluation Model. Kirkpatrick’s model is the second, go-to, quality item when I design and when I evaluate other’s design.

What do you evaluate, exactly? Here are the evaluations for each level:

• Level 1: Reaction — To what degree participants react favorably to the training.
• Level 2: Learning — To what degree participants acquire the intended knowledge, skills, attitudes, confidence, and commitment based on their participation in a training event.
• Level 3: Behavior — To what degree participants apply what they learned during training when they are back on the job.
• Level 4: Results — To what degree targeted outcomes occur as a result of the training event and subsequent reinforcement.

Important each level below builds ability to meet the next. You can not jump to Level 4: Results without accomplishing Levels 1: Reaction through Level 3: Behavior to note that you need to build Level 1, then 2, then 3, to get to Level 4.

The Kirkpatrick Model, 4 Levels of Evaluation, Level 1 and Level 2 – select picture for larger view

The Kirkpatrick Model, 4 Levels of Evaluation, Level 3 and Level 4 – select picture for larger view

How do you measure? Well, that is a post on survey development and measurement, that will need to wait.

Not What You Know, But What Is No

Some organizations rely on external training partners to deliver their training, the logic comes from expectation these experts are highly competent professionals and allow organizations a discernible training focus to mix and match based on needs.

Outsourcing training comes with risk: that they bill as trainers, does not mean they know training.  If 70% of learning happens on the job, we need to justify budget accordingly.

How can you tell a professional trainer is serious?

A very clear indication if you are working with a professional trainer is to ask if their methods include all four of Kirkpatrick’s levels for evaluation. Ask about their view of the 70:20:10 learning model.

If they say, “no”; move on.

If they say, “yes”, ask how they measure as well as a sample of some of their evaluations.

Keep a particularly discerning eye for bad Level 1 Evaluations that focus on the food quality served at training, if the instructor wore pastel colors, and if the temperature is comfortable.

It is not that you evaluate, it is what you evaluate.

You can not change what you can not measure.

The entire point of communication is to build confidence, those with confidence to learn have competence to contribute.

If you want to tuck into evaluation immediately I recommend trainingcheck.com as an excellent source for creating evaluations at different levels with samples and insight for each level.

Three, or Four, Great Learning Sources:

1. Dr. Don Kirkpatrick — author, consultant, and trainer at Kirkpatrick Partners
2. Barbara Carnes — author, consultant, trainer, and blogger at Making Training Stick
3. American Society for Training & Development ASTD 2012 State of the Industry Report
4. trainingcheck.com Guide to Training Evaluation