ROI-Return+on+Investment

=(Article:ROI in the public sector)= = = =Results-based Approach to Training= In a nutshell, here is how ROI works: Here's a basic formula for calculating ROI: 1. Collect level-4 evaluation data. Ask: Did on-the-job application produce measurable results? 2. Isolate the effects of training from other factors that may have contributed to the results. 3. Convert the results to monetary benefits. 4. Total the costs of training. 5. Compare the monetary benefits with the costs. The non monetary benefits can be presented as additional-though intangible-evidence of the program's success.

RETURN ON INVESTMENT (ROI)
For years, there has been debate about the appropriateness of ROI as a measurement tool for learning and development (L&D) initiatives. Today, the debate has shifted to the ways in which ROI can be integrated into other measurement processes. Much of the use of ROI has come from pressure from senior executives, who not only request more value but, in some cases, demand an ROI for certain programs in order to justify budget allocations. These requirements have led the L&D function to face reality: ROI is an issue that will not go away.

Over 1,000 organizations have taken the initiative to implement ROI, based on the number of organizations participating in a comprehensive certification process designed to help individuals and teams implement the ROI methodology. With the acceptance of ROI, much of the focus has now turned to best practices for ROI implementation. The following 11 best practices represent the state of the art with those organizations that have successfully implemented the American Society of Training & Development’s ROI program.

Best Practice #1.
The ROI methodology is implemented as a process improvement tool and not a performance evaluation tool for the L&D staff. L&D staff acceptance is critical for the implementation of this process. No individual or group is willing to create a tool that will ultimately be used to evaluate his or her performance. Consequently, many organizations recognize that ROI is a process improvement tool and communicate this posture early. The process shows not only the success of a particular project, program, or solution, but also provides detail into how the project can be revised to add additional value. Barriers and enablers to success are always identified.

Best Practice #2.
The ROI methodology generates a micro level scorecard with six types of data. These data points reflect six distinct levels, each with a specific measurement focus:

Level One: Reaction and planned action - Measures participant satisfaction with the program and captures planned actions Level Two: Learning - Measures changes in knowledge, skills and attitudes Level Three: Application - Measures changes in on-the-job behavior Level Four: Business impact - Measures changes in business impact variables Level Five: ROI - Compares program benefits to the costs Intangibles - Measures that are purposely not converted to monetary value

As shown in the list above, the data represents a scorecard of performance, representing both qualitative and quantitative data, often taken at different time frames and from various sources. This approach builds on Kirkpatrick’s four levels and generates a balanced, micro level scorecard for individual programs.

Best Practice #3.
ROI methodology data are being integrated to create a macro scorecard for the learning and development function. As more and more studies are conducted, data are rolled up to create a macro level scorecard, showing the value of the function. As shown in the figure below, the individual micro scorecard evaluation data are integrated into the overall macro level scorecard. This approach requires a few similar questions to be asked each time. These are then integrated, using technology to create the L&D macro level scorecard.

Best Practice #4.
ROI impact studies are conducted very selectively, usually involving 5-10% of all programs and solutions. Programs that are usually targeted for level 4 and 5 evaluations are those that are strategically focused, expensive, high profile, controversial, and reflective of management’s interest. This does not mean that other programs are not evaluated. It is recommended that all programs be evaluated at level 1 and the vast majority at level 2, but only a few select programs are taken to levels 3, 4, and 5. Most importantly, those involving the actual ROI calculation - taken to the 5th level - are evaluated at all five levels.

Best Practice #5.
ROI evaluation targets are developed, showing the percent of programs evaluated at each level. Organizations are targeting the desired number of programs to evaluate at each level, expressed as a percent. Target levels are developed reflecting the resources available and the feasibility of evaluation at each level.

Best Practice #6.
A variety of data collection methods is used in ROI analysis. ROI evaluation is not restricted to a particular type of data collection method such as monitoring of business data. Instead, questionnaires, built-in action plans, focus groups, and observations are used in developing the complete profile of six types of data in the ROI methodology.

Best Practice #7.
For a specific ROI evaluation, the effects of L&D are isolated from other factors. Although a difficult issue, best practice organizations realize there must be some method in place to show the direct contribution of the L&D program to make a business linkage to a specific L&D effort. Many best practice organizations are currently using a variety of techniques ranging from control group analysis to expert estimation to tackle this issue with each impact study. Some argue that this is too difficult or impossible to do. In reality, it must be done for executives to understand the relative contribution of L&D. Otherwise, there’s a temptation to slash the budgets of major programs because there’s no clear connection between the program and the business value.

Best Practice #8.
Business impact data are converted to monetary values. These days, it may not be enough to report L & D program outcomes as expressed numbers in quality improvement, cycle time reduction, turnover reduction, enhancement in customer loyalty or job satisfaction. The actual value in monetary terms is absolutely essential in developing ROI because an ROI calculation compares the monetary value with the cost of the program. Best practice organizations are using a full array of approaches to develop monetary values.

Best Practice #9.
The ROI methodology is being implemented for about 3-5% of the L&D budget.

One of the common fears of ROI implementation is the excessive cost in both time and direct funds. Best practice firms report that they can implement the ROI methodology for roughly 3-5% of the total budget, using appropriate evaluation targets discussed in #5. Available cost savings approaches include the following:

• Plan for evaluation early in the process • Build evaluation into the training process • Share the responsibilities for evaluation • Require participants to conduct major steps • Use short-cut methods for major steps • Use sampling to select the most appropriate programs for ROI analysis • Use estimates in the collection and analysis of data • Develop internal capability to implement the ROI process • Utilize web-based software to reduce time • Streamline the reporting process

When implementing ROI, many organizations have migrated from a very low level of investment (around 1% or less) to the 3-5% level by a process of gradual budget enhancements. These enhancements sometimes come directly from the cost savings generated from the use of the ROI methodology.

Best Practice #10.
ROI forecasting is being implemented routinely. Senior executives are sometimes asking for a forecast of ROI before a project begins. Consequently, best practice organizations are routinely using ROI forecasting approaches to enhance the decision making process. The credibility of the process is greatly increased by the use of conservative adjustments and built-in steps to secure input from the best experts.

Best Practice #11.
The ROI methodology is used as a tool to strengthen/ improve the L&D process. A significant payoff for using the ROI process over time is that it transforms the role of L&D in the organization. Application of the process increases L & D alignment with business needs, improves the efficiency of design, development, and delivery, and enhances the value of learning and development in the organization. Furthermore, it builds respect, support, and commitment from internal groups, including senior executives and major program sponsors. Source: American Society for Training & Development Links, September 2003 based on thoughts from Dr. Jack Phillips, an international expert on measurement and evaluation.