» E-mentor nr 3 (10) / 2005

If you listen to the hype, e-Learning trains staff better and has a massive and rapid Return on Investment (ROI).

But how accurate is this? The author puts forward the theory that, when taken under the microscope, e-learning is not meeting these expectations. In fact, employees are often down-skilled as opposed to up-skilled and the massive ROI is but based on an often deliberate misuse of an ROI methodology.

The author states that, at the heart of the e-Learning myth lies a restrictive calculation of real costs and investment together with assumption about learners' abilities and motivation to learn. In turn, accurate measurement of the didactic and financial success of the project is compromised if not lost.

The author proposes that this does not have to be the case: With effective RIO definition at the onset and a linking of learning objectives to business impact, e-Learning projects can leverage considerably more benefits for employees as well as organizations.

One of the boldest claims of e-Learning is that the introduction of e-Learning enables employees to be better trained and at the same time has a massive and rapid Return on Investment (ROI). In reality, this ROI is mainly accounted for by the reduction of the cost of training staff and employees by avoiding sending them away from their desk, reducing travel, administrative and instructors' costs. E-Learning also claims to allow learners to learn what they need when they need it. This is the just-enough, just-in-time and just-for-you concept. All these claims help fuel one of the largest myths: that e-Learning has a very fast ROI, and trains staff better.

We now know that e-Learning is not meeting these expectations, and that, in fact, there is currently a risk that e-Learning is not facilitating the up-skilling of employees, but is rather contributing to the non-training of the workforce. There are many reasons why this is happening, some of which are linked to the learning culture within organizations, to peer pressure, or, the fact that employees are at their desks to do productive work and not to learn. Other reasons are linked to the e-Learning offering not being aimed at the learning needs or goals of individuals and organizations. In fact, all the reasons for failure are mainly the consequences of an ill-defined program, where the Business Objectives have not been defined against the needs of the organization and where the definition of ROI and the measures of the program are not defined.

And so it is that, after all the hype of the last years, we now find that e-Learning does not deliver value for the money invested. In addition, there is even no way of assessing whether learners are learning more or better. In fact, we cannot assess the impact of the program against business indicators.

So is there a return on investment for e-Learning? Or is e-Learning just a way for training managers to reduce the cost of training? Is there a positive and measurable impact of e-Learning?

In this article, I will propose some guidelines for starting e-Learning programs in a way that the business indicators impacted are well defined, allowing for the measurement of the efficiency of the initiative, and the value of the program itself.

I will first make an assessment of e-Learning and then outline how ROI is defined today and then I will propose an approach and a set of recommendations for setting up Corporate e-Learning programs.

E-Learning today

As I mentioned earlier, e-Learning has been accompanied by a lot of hype. And many organizations have implemented massive and very ambitious e-Learning programs, thinking that this will help cut the cost of training their employees. Expectations were and still are very high. E-Learning has too often been defined as a way to bring learning content to the employees at the point of use (usually their desk), accessible at any time, anywhere and tailored for the learner.

Why does e-Learning fail?

So why does e-Learning fail? As mentioned earlier, the way e-Learning is currently introduced into organizations is top-down. It is also mainly an economical decision. There is little or no reflection on the business drivers to be impacted, or on the cultural changes and other pre-requisites for the introduction of this type of program.

There are many assumptions made, such as the motivation of the workforce to learn on their own, on the availability - in the case of on-line collaborative learning - of colleagues, peers, coaches or experts, on the culture of the organization. This last point refers to peer-pressure. Even if the organization would like (and sometimes defines an e-Learning charter) to allow time for employees to go on-line and learn, this time is, in practice, rarely available workforce (they can still do this learning after hours!). The need to deliver, to log hours against billed projects or clients is too often a higher priority for the organization. The same applies for the employees who do not understand that colleagues cannot help them. In reality, there is very little time made available to the employees to focus on their own learning.

So, there is no dedicated time set aside to help employees focus on what it is they need to learn, and then learn it. All e-Learning allows (or should I say dictates) is that employees define their own pace, their own learning curriculum. It is quite a big assumption to think that every employee in an organization can do this!

Finally, the e-Learning offering is neither linked to individual nor business goals. Too often, the decision to purchase a Learning Management Solution and the thousands of hours of unfocused e-Learning content supersedes the analysis of the needs. In all this turgid content employees do not find what they need when they need it. And needless to say in some cases, the boredom of sitting in front of a screen being subjected to non-appealing and non-specific content also contributes to non-usage.

There is one exception here, and I would categorize this as management mandatory training. This could be compliance training, or safety training, for which the organization has set a goal of 100% of staff completing the course. This goal is then measured on a regular basis. In this case, managers and team leaders are making sure that employees are accessing the course.

ROI and e-Learning today

Today, the ROI is defined using very simple rules. Usually, it is a combination of Cost savings versus an initial investment. In other words, the following methodology is used;

Where the cost savings are:

• how much will I save by not sending X employees that I need to train x cost of travel,
• cost of a trainer to deliver the training,
• cost of the administration for organizing training classes, preparation and booking rooms and venues, etc,
• cost of the time away of the office.

And investment is:

• the cost of a platform (Hardware, LMS or LCMS),
• the cost of content (courseware, SCORM Modules, etc),
• some marketing cost.

Based on this formula, a lot of e-Learning solutions have, at least on paper, a huge Return within a very short time. But unfortunately, this formula is very restrictive, and does not reflect of the real cost, such as the content creation, the sustaining and maintenance cost and all other attached licensing, server monitoring and upgrade costs, and even the program cost.

Also, by using this formula, the only real benefit of using e-Learning is that learners can access the content anywhere. There is no mention of any other benefit (such as the learning effectiveness, or business impact, or motivation, etc).

This formula is usually misused on purpose: to demonstrate that there is a big ROI, and to justify the strategic need for the organization to go in this direction. In fact, as this is a top-down decision, and the sole purpose is to reduce training cost, this approach is very useful.

• sustaining costs:
  • or the content, and the learning modules,
  • for the management of Metadata, and other tagging and classification/navigation data,
  • for the platform, such as the upgrade of the Operating System, Antivirus, Backup, new versions, Database, etc,
• administration costs,
• internal marketing cost: this is the program around the internal advertising, coaching, support, etc.

In conclusion, e-Learning, as we know it today, is not meeting all the claims that it is making. E-Learning is contributing more to non-training of the organization. This could impact an organization quite badly, as it can result not only in a down-skilled workforce, but also in poor results and projects for Clients, of even in the de-motivation of the employees. In fact, today, e-Learning is mainly helping e-Learning specialized companies to be prosperous, and to develop their markets.

E-Learning can also be quite expensive: developing customized programs or content can be very efficient, but also very costly. Maintaining the learning management system can also result in high cost.

So should companies run away from e-Learning: this expensive tool that does not help train their employees to help the company maintain a distinct competitive advantage?

In fact, some of the benefits claimed for e-Learning could be leveraged. Combined with the business-driven approach and linked with the business drivers and goals, correct organizational changes and supported by an appropriate program, e-Learning could deliver a lot of benefits for an organization, as well as a ROI. The organization would need to understand that the ROI would be linked to the business objectives, and the shifting of business drivers, that e-Learning needs a substantial focused investment (i.e. not in the technical environment, but in the organization and the content), and that the ROI will not be based soley on the cost savings.

This is the approach I would like to elaborate on in the rest of the paper.

E-Learning with a ROI

The definition of a ROI and the measures for success are required for any project. For example, when an IT director wants to implement a new Front End Solution costing several tens of thousands of Euro, he/she will need to justify this investment, make a business case supported by added value for the business and by a strong ROI. Unfortunately, this is not the case in corporate e-Learning projects, and this is the main gap, as it does not allow measuring the success and the efficiency of the program.

The impact of E-learning on work practices and the organization itself needs to be measurable and there must be verifiable sets of both tangible and softer benefits. So it is important to define the ROI and the way that the efficiency of an e-Learning program is measured at the very start of the definition of the actual program. In other words, it is at the definition and exploration phase that the objectives, outcomes and measures are implemented.

ROI for e-Learning can be defined, so that there is a way to ascertain whether that the program is making the expected impact, and also to quantify the benefits for the employees and the organization. I will suggest a method to define the ROI, and give some guidelines to define e-Learning projects, and also to select only the ones that have a definitive ROI, and keep conventional training methods for other development needs.

This way of defining e-learning projects is innovative and the results go far beyond the standard cost savings arguments.

The proposed approach

The proposed approach is based on the Kirkpatrick model and the Phillips extension. The Kirkpatrick model is widely used by Training and Organizational Development groups in organizations in order to evaluate the efficiency of courses and training programs. This model is very powerful, but it is not used to its full capacity today. In fact, the model is mainly used for levels 1 and 2, which translates into the use of happy sheets as evaluation, attendance to a session, and some basic questions about the usefulness of the training. In practice, most attendees fill in that the course was useful, pertinent and good, as they do not want to be caught in follow-up sessions, or they do not want to be seen as unpleasant. There is no evaluation or assessment of either learning objectives being met, or business impact.

The Phillips extension allows quantifies the business impact on the organization, and therefore measures the benefits that the learning or training program ibrings to the organization in monetary terms. As this is very dependant on the evaluation of the Kirkpatrick levels, this model is very rarely used in organizations.

Here is the model that I will use for the rest of the article. It is not the original Kirkpatrick and Phillips models. For more information, the reader should analyse the original model.

Typically, people try to use this starting with level 2, where the learning objectives are defined. It is true that if a learning or a training program needs to be defined, it is easier to structure it around the concept of modules or classes, and therefore around the Learning Objectives. The others are derived after this (with more or less success). The Business Objectives are tied to the Learning Objectives, and may therefore not be relevant for the success or the growth of the organization.

The trick here is to use the model differently. In fact, it is to start with what is important for the organization and for the business goals, and therefore to start with the Business Indicator we want to impact.

So here is the approach:

1. Definition of the Business Objectives to be impacted: this is the definition of what we want the organization to do differently, and which business drivers need to be shifted. These can be prioritized or weighted against the vision of the company, or used to create a distinct business advantage, or simply to catch up with the competition.
2. From these business objectives, define Level 3: What do we need employees to be able to do in order to achieve the impact these business objectives? By doing this, we define the proficiency of employees to perform specific tasks. Some business objectives as defined for level 4 may not be impacted by having employees perform specific tasks, and are therefore discarded for the rest of the analysis.
3. Once we have identified the Application Objectives, we can derive the Learning Objectives: What do employees need to learn in order to meet the application objectives? At this stage of the analysis, we can select the application objectives that can be acheived by meeting specific learning objectives. As for the previous step, some application objectives may not be met by Training or by the knowledge. These Application and associated Business Objectives will then be discarded from the analysis. An analysis will also have to be performed at this stage to assess what is the best way to meet these learning objectives: some will be by e-Learning, other by on the job Training, classroom based, a blended approach or by seminars. E-learning is not the answer for all the learning objectives, and only some of the objectives can be, either totally or partially, met by e-Learning.
4. And finally, for the business objectives that are still eligible for the program, we can quantify the gain for the organization.

The result of this analysis could derive the following program charter: By implementing this Learning program, we want employee to learn X, Y and Z so that they can do the following tasks impacting the following business drivers that will result in an estimated value of Єxxxx.

This is the mission of the program! This is what will be communicated to the stakeholders of the program! This is what is going to be communicated to the employees ... to the community of Learners! By doing this analysis, we are able to link the learning objectives up to the business impact and therefore to the value that this will generate.

Everyone understands the reason for this program and the benefits to them of learning from the material in the program. Peers, colleagues and management also understand the benefits, and therefore facilitate the learning effort, and respect the time allocated for the learning.

The cost aspect

As for any project, there is an investment to be made. In e-Learning projects, the investment is typically around the infrastructure, both Hardware and Software, and on the acquisition of content. I think that it is important to remind you that e-Learning can be costly, and that the focus should not be on the delivery of the content, but on the content, and the program itself.

Sustaining and maintenance costs can be quite expensive, and it is important not to forget these when defining the ROI plan over several years.

Conclusion

E-Learning is a fantastic tool for helping organizations to meet new business goals, and create or keep a distinct business advantage. When well-defined, the e-Learning programs can leverage benefits for the learners and for the organization. The e-Learning concepts can be maximized.

But, as with many technological solutions, the key resides in the definition of the program, and the link to the business objectives, goals and results. Therefore, the definition of the program must be carried out around these business aspects, so that the program's missions and goals are obvious and easy to communicate.

It is only in this case, i.e. when the program is linked to a business-driven ROI, and measured against this, that you will be able to implement a program that has a positive impact.

I would like to recap some of my recommendations:

• There is no silver bullet. The ROI must be linked to the business objectives.
• Invest in the definition of the business goals and the program. Communicate at every level of the organization.
• Invest in the content, not in the infrastructure. Maybe start with Open Source Software (for example, Moodle, www.moodle.org).
• Ask for help from experts and consultants, preferably small and flexible consultancy organizations that do not have off-the-shelf solutions.
• Create the program, define the plan and stick to the plan.
• Measure and Communicate.