Sunday, November 19, 2017
Return on Expectations Minimize

Return on Expectations (ROE) is rapidly sweeping the learning and development industry. The reason is quite simple: when used properly, it never fails to show the value of training in the terms desired by key stakeholders.

Kirkpatrick Partners stresses the value of ROE over the traditionally measured Return on Investment (ROI) because ROE is designed and executed in partnership with the people who will determine learning's value.

The ASTD's August 2010 edition of T & D magazine contained Jim and Wendy Kirkpatrick's article "ROE's Rising Star: Why Return on Expectations Is Getting So Much Attention."

Read the full text of the article below, click on the image to the right for a PDF copy, or click here to read the article on ASTD's website.

 Click here to read the article 'ROE's Rising Star: Why Return on Expectations Is Getting So Much Attention,' written by Jim Kirkpatrick and Wendy Kirkpatrick.

Did you enjoy this article?
If so, complete our free
registration to gain access to 70+ articles, podcasts and other resources in our Resource Library.
ROE's Rising Star Minimize

ROE's Rising Star: Why Return on Expectations Is Getting So Much Attention

Return on expectations (ROE) is rapidly sweeping the learning and development industry. The reason is quite simple: when used properly, it never fails to show the value of training in the terms desired by key stakeholders. This is because ROE is designed and executed in partnership with the people who will determine learning's value. The Kirkpatrick Foundational Principles explain the ROE process and the critical importance of business partnership in achieving it:

      • The end is the beginning.

      • ROE is the ultimate indicator of value.

      • Business partnership is necessary to bring about positive ROE.

      • Value must be created before it can be demonstrated.

      • A compelling chain of evidence demonstrates your bottom line value.

The end is the beginning

Showing the value of training begins before the program even starts. Donald L. Kirkpatrick says it best in Evaluating Training Programs: The Four Levels:

"Trainers must begin with desired results and then deter- mine what behavior is needed to accomplish them. Then trainers must determine the attitudes, knowledge, and skills that are necessary to bring about the desired behavior(s). The final challenge is to present the training program in a way that enables the participants not only to learn what they need to know but also to react favorably to the program."

The clearer a training professional is on the desired Level 4 results of a solution (See Table 1), the better chance that his time and effort will cost-effectively accomplish that end. A common error many training professionals make is defining desired results at the level of training instead of relating training to the accomplishment of the highest level business goals. Here are some common examples of true Level 4 results:

      • sales

      • employee retention

      • profitability

      • saving or extending lives

      • prevention of damage to the environment.

It is also critical for the training professional to find out how important the initiative is to the organization and obtain some idea of the resources that can be allocated to it. Not every single initiative and program will warrant a complete execution plan as outlined in this article. The level of effort should be in alignment with the importance of the initiative to the success of the organization.

Table 1
Level 4: Results  
To what degree targeted outcomes occur, as a result of the learning event(s) and subsequent reinforcement. 

Level 3: Behavior       
To what degree participants apply what they learned during training when they are back on the job.
Level 2: Learning
To what degree participants acquire the intended knowledge, skills and attitudes based on their participation in the learning event.

Level 1: Reaction


To what degree participants react favorably to the learning event.


ROE is the ultimate indicator of value

When executives ask for new training, many learning professionals retreat to their departments and start designing and developing suitable programs. Sometimes, a training needs assessment is conducted, but it is rarely taken beyond training and into critical performance and business needs. This is where a lot of training waste occurs: training that isn't needed is conducted, and the required training or other solution that would be truly beneficial is not even identified.

Stakeholder expectations define the value that training professionals 
are responsible for delivering. Learning professionals must ask the stakeholders questions to clarify and refine their expectations on all four Kirkpatrick levels, starting with Level 4. This is a negotiation process in which the training professional makes sure that the expectations are satisfying to the stakeholder and realistic to achieve with the resources available. Learning professionals then need to convert the typically broad, unquantified expectations into observable, measurable Level 4 results by asking the question, "What will success look like to you?" These Level 4 results become the targets on which collective efforts are focused.

Next, learning professionals should work with the managers of the intended training participants. Together they should identify the critical behaviors of training graduates, which, if performed consistently on the job, will most likely produce the desired results. Only then can the training professional proceed with the traditional identification of learning objectives. 


At the same time, the training pro
fessional should ask what evidence will be required at each of the four levels to show that the initiative was a success. When the measurement methods, tools, and techniques are defined at the start of the initiative, it makes evaluation much easier to perform. 

Business partnership is necessary to bring about positive ROE

Once there is a clear understanding of the result to be accomplished, the next step is for training professionals to work with business managers and supervisors to create a tactical execution plan. This is necessary because training events alone typically result in only 15 percent transfer of learning to on-the-job behavior, according to Robert Brinkerhoff.


Historically, learning professionals have focused most of their efforts on training events (Levels 1 and 2); however, the largest ROE occurs at Level 3, on the job. For maximum ROE, there needs to be agreement about the intensity of effort required from all departments before, during,and particularly after training. Specifically, learning professionals need to partner with supervisors or frontline leaders

      • at checkpoints during training design
      • before training to prepare participants
      • after the training to support and monitor trainee performance on the job.

When participants return to the job after training, required drivers—processes and systems that reinforce, monitor, encourage, or reward performance of critical behaviors on the job—must be in place to provide both support and accountability for the consistent performance of critical behaviors. The degree to which drivers occur directly relates to the extent to which critical behaviors are performed. Performance of critical behaviors is what yields business-level results. Results determine the ROE from the initiative.

Value must be created before it can be demonstrated

At this point in the process, desired results are clearly defined, and a cross-functional execution plan has been created. What remains is the implementation of the plan.


Organizations spend more than 72 percent of their training evaluation re
sources measuring Levels 1 and 2 (ASTD, "The Value of Evaluation" report). This shows how much money training professionals are spending on employee training. It leaves just 28 percent of the training evaluation budget for demon- strating how the training supports on- the-job behavior and accomplishment of key business results (Levels 3 and 4).

Seventy percent of the time, when participants fail to consistently perform desired behaviors on the job, it is due to factors in the work environment (2006 ASTD State of the Industry report). It is only 10 percent of the time that poor performance is due to something that did or did not happen during formal training.

Very little time and attention is being given to the critical on-the-job application environment for new knowledge and skills. To create maximum ROE, it is essential that learning professionals do everything possible to partner with supervisors to support on-the-job behaviors and executing the required drivers. To accommodate this, fewer resources should be spent on Levels 1 and 2 to allow more training involvement in Level 3.

While monitoring and encouraging 
on-the-job behavior, training professionals should create an "early warning detection system" in which preliminary measurements of identified metrics are taken. Areas that are not hitting targets should be identified so that adjustments to the plan can be made. This keeps major long-term initiatives on track and creates the highest likelihood that the desired results will be achieved.

A good way to communicate the status of critical behaviors, required drivers, and leading indicators (short-term observations and measurements that suggest that critical behaviors are on track to create a positive impact on desired results) is with a dashboard. A dashboard is a visual depiction of goals set for each of these items and the current status of each. Dashboards are a highly effective driver and one that executives favor.

A compelling chain of evidence demonstrates your bottom line value

Training is on trial, accused of failing to deliver sufficient business value to the bottom line. There is an urgent need for training professionals to demonstrate the value they do bring to the business by presenting a chain of evidence at the end of important initiatives.

This process is easy and fun when the Kirkpatrick Foundational Principles are followed because the evidence to be collected is clearly defined at the beginning of the project by the stakeholders, and data is collected all along the way. It can be very rewarding to assemble the story of how working together in business partnership has created the desired ROE.

The chain of evidence includes 
both numeric data and the testimonials and stories of those who participated in the process. This combination of evidence shapes a story that appeals to and is understood by everyone. Training professionals should prepare a compelling presentation that will be delivered in person if possible, to share this data with stakeholders and the entire project team. 

Telling this story will earn train
ing professionals a "not guilty" verdict for the accusation of failing to deliver adequate value for their efforts. Since the plan was created and executed in partnership, the training professional is then cast in a light of delivering a positive story of success.

"But we were asked to show our ROI"

It is common that training professionals are asked to "show their ROI." Return-on-investment has become shorthand for value much the way "Kleenex" has become the accepted name for facial tissue. Be sure to ask the stakeholders exactly what sort of measurement of value they require. Generally what ensues is a conversation in which the stakeholders say they want to know what they are getting for the time, money, and resources invested in learning and development efforts.

Some well-meaning training profes
sionals don't ask questions and just assume that an ROI study to isolate the impact of training is required, and proceed to spend even more money to hire someone to deliver it. It is important to clarify exactly what ROI is to understand how it differs from ROE. ROI is defined as the ratio of money gained or lost (whether realized or unrealized) on an investment, relative to the amount of money invested.


Net Program Benefits × 100 = ROI% 
Program Costs

How ROE differs from ROI

Table 2 indicates the key differences between ROE and ROI. Brinkerhoff comments on the use of ROI in measuring the value of training

as follows:

"To isolate the impact of training is methodologically impossible and strategically counterproductive. If we're trying to build partnerships with managers of trainees, we can't use an evaluation plan that isolates the impact of training alone and ignores the critical management support after the training event. It's like saying I've got a new laptop; now let's see if we can isolate the value of the B key." 

Table 2
ROE ROI
Proactive, business partnership approach that unifies team 

Defensive isolation approach that separates functions

Defines training as a contributor to key business results

Defines training as an end in itself

Value defined by business stakeholders in cooperation with training                                                    
Value defined by predetermined formula

Focus on comprehensive evidence and a compelling story of value
Focus on single metric of numeric proof

Easy to understand, flexible and cost effective Complex, rigid and expensive


Getting Started

Get started on building ROE in your organization today:

      • Identify the key stakeholders that judge the "success" of training.

      • Get to know your stakeholders and identify their priorities.

      • Create a cross-functional advisory committee with representatives from business and training.

      • Select a strategic initiative, and use the ROE process.

The reality is that companies are more likely to let a "contractor" go before a member of their own team. Earn your place on the team by being a cooperative member who works collaboratively towards the overall success of important initiatives. Then watch the confidence in your work grow and requests to "prove the value" decrease.

PricewaterhouseCoopers and ROE


In April 2010, Jim Kirkpatrick delivered a Kirkpatrick Four Levels Evaluation Certification Program in Vancouver, British Columbia. The group was treated to a conversation between two PricewaterhouseCoopers leaders: Eric Andrew, national private company services leader for PwC Canada, and Deana Gill, a program attendee and national PCS learning and development senior manager. Both strongly acknowledged that they are indeed strategic business partners, and that the success in Eric's area was only possible through close collaboration and an integrated team approach. 

One of the other program participants asked Eric, "What do we have to do to get the attention of and buy-in from senior business leaders like you?" Without pause, he answered, "Show me how what you do contributes to my results." Eric and Deana then led a lively discussion, with ROE taking center stage. 

It was determined that ROE must begin with a business partnership mentality and approach. This involves the identification of the specific "needles" (business and human resource metrics) that must move in order for the initiative to be a success. This approach must also include agreement on the level of effort (in addition to training) required to bring about those results.

After agreeing on the specific metrics and execution plan, training professionals (such as Deana) and the identified program champions make it happen. They then need to present the chain of evidence to senior leaders (such as Eric) showing how these efforts contributed to the overall result.

About the Authors Minimize
Wendy Kirkpatrick is the founder of Kirkpatrick Partners, a company dedicated to helping organizations become more effective through business partnership. She applies her skills as a certified instructional designer and expert presenter and facilitator to lead companies to measurable success.

Click here to read Wendy's full bio


Jim Kirkpatrick, PhD, is a Senior Consultant for Kirkpatrick Partners. His major area of expertise is the Kirkpatrick Business Partnership Model. Jim consults for Fortune 500 companies around the world, including Harley-Davidson, Booz Allen Hamilton, L’Oreal, Clarian, Ingersoll Rand, Honda, the Royal Air Force and GE Healthcare. Jim is a masterful facilitator and conducts workshops and keynote presentations around the world on the Kirkpatrick Model and business partnership.

Click here to read Jim's full bio

Join the Kirkpatrick Community

Register with us for FREE samples of our Blended Evaluation Plan® form.

(If you have already registered with us, simply log in and visit the Resources tab to access the samples - no need to re-register.)
 


You'll also receive immediate access to our Resource Library, and weekly updates from Kirkpatrick Partners.



Click here for more information about the materials available in our Resource Library and to see a sample of our weekly newsletter.
Learn More
Click here to learn how to maximize results using the Kirkpatrick methodology.                             

Visit the Training & Events page to see upcoming Kirkpatrick training programs and public appearances.
        
   

Visit the Products page to see all of the available Kirkpatrick books and other materials, including our latest book, Kirkpatrick's Four Levels of Training Evaluation

Copyright 2009 - 2017 Kirkpatrick Partners, LLC. All rights reserved.