Blended Evaluation: Moving from the Past to the Future
Last week, we discussed possible sources of data and methods for gathering it when embarking upon blended evaluation. Click here if you missed it.
This week, we continue our conversation about hybrid approaches by busting one of the most common beliefs in the world of evaluation in general, and about the Kirkpatrick Model specifically.
Read on to find out what this common misconception is, and how to make an initial break from it using a free Kirkpatrick tool.
The extremely counterproductive belief that has been standard practice for decades is that each Kirkpatrick level should be assessed individually. This belief usually results in L&D professionals distributing post-course surveys, commonly called “evals,” “smile sheets” or even “Level 1s,” which only occasionally address issues beyond the immediate participant reaction to the training and the trainer.
At the end of a program, we often see “Level 2s,” which most commonly take the form of knowledge tests or perhaps skills tests. Again, there is little or no connection to any other levels. “Level 3s,” if they occur at all, typically take the form of a survey sent to training grads and their supervisors 60 or 90 days following the completion of a formal program. While it is good that the bridge between training and the business is being crossed in these cases, evaluation at this level often amounts to little more than asking, “How are things going over there?”
Finally, “Level 4s” are often non-existent because those without a working knowledge of the New World Kirkpatrick Model are unsure of how to assess Level 4.
The bedrock of the New World Kirkpatrick Model is that the levels are best assessed collectively, not separately. You are probably familiar with the meaning of the word hybrid. Notice the image of a hybrid rose in this post. The definition of hybrid in our world is, “Tools that measure multiple levels at the same time, such as a survey with questions related to both Levels 1 and 2 immediately following training.”
One example of hybrid evaluation is an instructor observing a small group during classroom activity, looking for each individual participant’s level of involvement (Level 1) and ability to demonstrate certain skills (Level 2). At the end of the activity, the instructor “pulls up a chair” and leads a discussion on how they might apply what they just did on the job (Level 3) and how their customers will likely respond to their efforts (Level 4).
Another example would include the following focus group questions targeted to a group of supervisors in regards to training their employees completed two months prior:
1. What have you seen your employees doing differently over the past 2 months? (Level 3)
2. To what can you attribute their improvement? (Levels 2 and 3)
3. What signs of success (Level 4) have you seen from their efforts (Level 3).
A final example of hybrid evaluation can be found within the Kirkpatrick hybrid survey tool. Note that the items within this tool address Levels 1 and 2 directly, and intentions and expectations for Levels 3 and 4. You can access the hybrid survey tool on our Resources page under Kirkpatrick® Diagrams and Forms; just log in to the site, or create an account if you’ve never signed up with us before.
Additional Resources
Kirkpatrick Four Levels® Evaluation Certification Program – Bronze Level