Beware the Risks of One-Dimensional Evaluation

Last week, we introduced the concept of blended evaluation.

Click here if you missed it.

This week, we will delve further into possible sources of data and methods for gathering it.

How do you move beyond just gathering data from the training participants? Read on to find out.

Consider this key concept: Truth gathered through blended evaluation leads to trust from stakeholders and line leaders.

Why doesn’t traditional evaluation lead to trust? It is often selective or one-dimensional, coming from a singular source instead of a rich variety of sources. Therefore, it tends to uncover only partial truth, at best. At worst, it could even yield untruths due to its sometimes biased, training-centered nature.

Last week, we discussed how to determine what evaluation needs exist. After determining what qualitative and quantitative data are required to serve specific evaluation purposes, the source of that data should be decided. Consider options other than traditional participants (yielding data at all 4 Kirkpatrick levels) and their supervisors (yielding data at Level 3). The table below provides examples of other potential sources and indicates which Kirkpatrick level that source may allow you to evaluate:

Source	Level 1	Level 2	Level 3	Level 4
Instructors	X	X
Dedicated observers	X	X
Classmates	X	X
Peers / colleagues			X	X
Direct reports			X	X
Help desk and on-demand training data			X	X
Department and organizational performance data (KPIs)			X	X
Department and organizational human resources data (engagement, retention)			X	X
Customer, client, patient data (satisfaction, outcomes)			X	X

After determining the sources you will use to gather your data, consider the methods you will use. The idea is to use whatever sources will be the least intrusive to the people being assessed, and still yield the most useful and accurate data possible.

The table below shows potential evaluation methods and the Kirkpatrick levels that they can evaluate:

Methods	Level 1:Reaction	Level 2:Learning	Level 3:Behavior	Level 4:Results
Survey, questionnaire, individual or group interview	X	X	X	X
Action plan monitoring, action learning		X	X	X
Work review, skill observation, behavior observation, action learning		X	X
Case study, knowledge test, knowledge check, presentation, teach back		X
Request for validation			X	X
Key business and HR metrics				X