Beware the Risks of One-Dimensional Evaluation

Last week, we introduced the concept of blended evaluation. This week, we will delve further into possible sources of data and methods for gathering it. How do you move beyond just gathering data from the training participants?

Consider this key concept: Truth gathered through blended evaluation leads to trust from stakeholders and line leaders.

Why doesn’t traditional evaluation lead to trust? It is often selective or one-dimensional, coming from a singular source instead of a rich variety of sources. Therefore, it tends to uncover only partial truth, at best. At worst, it could even yield untruths due to its sometimes biased, training-centered nature.

Last week, we discussed how to determine what evaluation needs exist. After determining what qualitative and quantitative data are required to serve specific evaluation purposes, the source of that data should be decided. Consider options other than traditional participants (yielding data at all 4 Kirkpatrick levels) and their supervisors (yielding data at Level 3). The table below provides examples of other potential sources and indicates which Kirkpatrick level that source may allow you to evaluate:

Source Level 1Level 2Level 3Level 4
InstructorsX  
Dedicated observersX  
ClassmatesX  
Peers / colleagues    XX
Direct reports    XX
Help desk and on-demand training data    XX
Department and organizational performance data
(KPIs) 
    X X
Department and organizational human resources data
(engagement, retention)
    X X
Customer, client, patient data (satisfaction, outcomes)   XX

After determining the sources you will use to gather your data, consider the methods you will use. The idea is to use whatever sources will be the least intrusive to the people being assessed, and still yield the most useful and accurate data possible.

The table below shows potential evaluation methods and the Kirkpatrick levels that they can evaluate:

MethodsLevel 1: ReactionLevel 2: LearningLevel 3: BehaviorLevel 4: Results
Survey, questionnaire, individual or group interview  XXXX
Action plan monitoring, action learningX XX
Work review, skill observation, behavior
observation, action learning
 X X
Case study, knowledge test, knowledge check,
presentation, teach back
X 
Request for validation   XX
Key business and HR metrics  X


Don’t Miss a Feature

Register with us to receive Kirkpatrick Quick Tips delivered to your inbox.