Who Says You Can’t Get There From Here? From Level 2 to Level 3
Throughout May, we remember Don Kirkpatrick and celebrate his contributions to the training industry. Enjoy this legacy piece penned by Don. The concepts are timeless and applicable even today.
Who Says You Can’t Get There From Here?
From Level 2 to Level 3
By Donald Kirkpatrick, Ph.D.
In case you aren’t familiar with them, I’d better start out by defining the Four Levels of Evaluation. In 1959, I wrote a series of articles for the journal of the American Society for Training and Development (ASTD). The articles were based on my Ph.D. dissertation at the University of Wisconsin. The subject was “Evaluating a Human Relations Training Program for Supervisors.”
The articles focused on the four different levels of evaluation that I had used to evaluate the training program under discussion. Level 1 was Reaction; Level 2 was Learning; Level 3 was Behavior, and Level 4 was Results. I described the Four Levels as follows:
- Reaction: Evaluating the reaction of the participants to the program. I called it a measure of “customer satisfaction” because the participants were our customers.
- Learning: Evaluating the increase in knowledge, increase in skills, and/or change in attitudes.
- Behavior: Evaluating the change in on-the-job behavior because of attending the program.
- Results: Evaluating the final results from the program including increase in profits, increase in sales, reduction in costs, reduction in turnover, improvement in morale, and return on investment (ROI).
The articles attracted a great deal of attention, and training professionals began to refer to them as Level 1, etc. The word spread and trainers started to call the levels Kirkpatrick’s Four Levels. I was asked to speak at the conferences of ASTD, Training magazine, Linkage and other for-profit and non-profit organizations. At Ford Motor Company, one participant said that I had taken the elusive term “evaluation” and put it into four practical and simple steps so that trainers could begin to speak the same language.
It wasn’t until 1993 that I wrote the first edition of Evaluating Training Programs: The Four Levels because Jane Holcomb, a professional training friend, told me that trainers couldn’t find a copy of the 1959 articles. With this book, word spread around the world and trainers everywhere began quoting Kirkpatrick’s “Four Levels.” The second edition was published in 1998 and was translated into Spanish, Polish and Turkish. The third edition will be published in November 2005.
This is a very long introduction to the title of this article but provides the necessary background.
One of the greatest challenges for trainers is the transfer of Learning to Behavior, and trainers today are having trouble getting it done. One of the main reasons for this is that trainers have no control over behavior and results. They may have influence, but not control.
In this article, I will describe one of the most important conditions that must be in place before transfer can take place. I call it “the proper climate.”
By “climate” I am referring to the boss of the participants in the training program. After the training program is over, each participant goes back to the job (one hopes) with a desire to apply what has been learned.
There are five possible climates that participants face when they do so. These include the following:
- Preventing: The boss tells the subordinate to forget about the training and get back to work. “Don’t forget that I am running this department and you will do what I tell you. You may have learned all about ‘diversity,’ empowerment,’ and ‘team-building,’ but forget it — you will do things my way!”
- Discouraging: The boss says to forget about the training and just get back to work. She might even add discouraging words such as, “Those trainers have never worked on the line and what they teach is a lot of theory.”
- Neutral: The boss doesn’t refer to the training at all and assumes that if the participant has learned anything useful, it will be applied on the job. The training is not discussed.
- Encouraging: Before the training, the boss sits down with participants and encourages them to learn any new ideas that can be used on the job. The boss tells the subordinates, “When you return, I want you to tell me what you learned and how I can help you apply it.” After the program is over, the boss takes the initiative and sits down with the participants and discusses what ideas the subordinates have and how they can work together to apply them.
- Requiring: In rare cases, the participant will be asked at the end of the class, “What ideas are you going to apply when you return to the job?” The boss gets a copy and sees to it that the ideas are applied.
My suggestion for ensuring that Learning is transferred to Behavior is to be sure that the boss is an “encouraging” boss. Before the program, the boss should sit down with the participant and go over the program, communicate why he or she is attending, and encourage note-taking of ideas that can be applied on the job. When the program is over, the boss should initiate the discussion and work with the subordinate to see that application takes place.
A number of companies are doing this. More and more, training professionals are communicating with the bosses of their trainees before a program begins and encouraging bosses to talk with subordinates about the program and encourage them to bring back new ideas.
Trainers should know bosses well enough to understand the climate they are likely to create for the participants when they return to the job. In addition, understanding why bosses are “preventing” or “discouraging” transfer from taking place is the first step toward making them become more “encouraging.” The most common reason for negative attitudes? Bosses have not been adequately involved in training program preparation and content preparation and/or they have not been informed of the purpose and content of the program.
George Odiorne, a writer of many management books, hit the nail on the head when he said, “If you want people to accept and support what you do, give them a feeling of ownership.”
My challenge to you is to use your influence (because you have no control over what happens on the job) to help create the kind of management climate that will encourage participants to learn what they can and apply it on the job — in other words, to transfer Learning to Behavior.
Join the Discussion
What types of bosses exist in your organization? How are they helping or hindering your programs? Share your input and experience via any of the following.
Don’t Miss a Feature
Register with us to receive Kirkpatrick Quick Tips delivered to your inbox each Wednesday.
Kirkpatrick Four Levels® Evaluation Certification – Bronze Level
Kirkpatrick Then and Now
Evaluating Human Relations Programs for Industrial Foremen and Supervisors
Celebrating Don Kirkpatrick’s Legacy
Evaluating Training Programs: The Kirkpatrick Four Levels DVD
Don Kirkpatrick Commemorative Package