How To Determine The ROI Of eLearning
The measurement of ROI of eLearning needs an integrated approach that should begin during the Training Needs Analysis or TNA phase and should successively build up right up to the determination of its impact on business.
How Do You Begin The Exercise To Measure The Effectiveness Of Online Training?
At EI Design, we use the approach shown here. Essentially, we focus on each stage from TNA to ROI calculation as the right action at each stage will impact the ROI of eLearning positively.
- Training Needs Analysis (TNA)
This is the baseline and will be eventually used to measure the desired gain for the learners (acquisition of new skill or fixing a gap) and if this gain resulted in the required impact the business had sought. Capturing the needs of the learners and having clarity on how you will measure their progress (that is, the expected gain) is a key component of the exercise. - Determining the right training format (online, blended, or ILT)
The next step is identifying the right format of training that aligns best with the TNA (Sometimes, training may not be the answer, and you need to identify supporting measures like coaching or mentoring). The selection of the right format is crucial in encouraging the learners to pursue it and also in ensuring that they connect with it, complete it and apply the learning on the job. - Identifying the learning strategy
The selection of the right learning strategy format is vital in engaging the learners. As we know, only an effective learning strategy can create a sticky learning experience. However, it is equally important to note that having formal training alone may not be enough. We also need to provide room for application of knowledge and practice or nudges to mastery. Hence, the learning strategy must have a combination of formal training and performance support intervention to successfully manage your mandate. - Determining the gain by the learners
An effective assessment strategy is the main approach to determine this. However, it is important to ensure that the measurement of this gain also factors for the validation of the application of the acquired learning and is not limited to the validation of knowledge acquisition. - Assessing the gain for business
We can look back at the parameters identified during the TNA stage and assess the required gain that has occurred and if the business saw the required impact. - ROI determination is the final outcome of this exercise
Once we monetize the gain and compare this with your costs, we will arrive at the ROI of your eLearning. Often, this outcome may need you to go back to the TNA and reassess or tweak the way forward.
To show how these aspects are interconnected and are part of a cycle is captured in the diagram here.
ROI determination methodology: One of the popular models used for ROI determination is the Kirkpatrick’s model of training evaluati on. I will outline what it entails and how can it be used to determine training effectiveness and the ROI of eLearning.
What Is Kirkpatrick’s Model Of Training Evaluation?
Kirkpatrick’s model of training evaluation is one of the popular models used to evaluate the effectiveness of training. It was created in 1959 and has undergone revisions in 1975 and 1994. The fact that it has survived for over half a century affirms the value it continues to offer.
It features the following four levels:
Level 1: Reaction
Level 2: Learning
Level 3: Behavior
Level 4: Impact
Initially, the model was viewed as a pyramid with each level building up from the previous one.
Increasingly, the same 4 levels are viewed as a "chain of evidence," and I feel this reflects a more relevant connection between the levels.
How Can You Practically Use The Kirkpatrick Model Of Training Evaluation To Determine The ROI?
To use the Kirkpatrick model of training evaluation, we need to identify two aspects at each level namely:
- What are we measuring?
- How will we use this outcome (to improve training effectiveness and increase its impact)?
Level 1: Reaction
Objective: At this level, the focus is to determine the learner’s reaction to the training. Today, we have wide-ranging options through Learner Analytics to identify if the learners liked the training if they found it useful and if they would be able to apply the learning.
From an evaluation perspective, this feedback enables L&D teams to assess if they are on track or if any further changes are required.
Level 2: Learning
Objective: At this level, the focus is to determine what was learned or gained (this should be attributable directly to the training).
From an evaluation perspective, this feedback enables L&D teams to assess if they met their mandate (captured during TNA) that could include:
- Knowledge gain.
- Acquisition of a new skill.
- Further proficiency gain on an existing skill.
- Behavioral change.
The pointers from this stage of evaluation would point to:
- The need for further training.
- The need to supplement formal training with other measures that could include performance support intervention or mentoring/coaching.
Level 3: Behavior
Objective: At this level, the focus is to determine if the learner behavior changed (again, this should be attributable directly to the training).
From an evaluation perspective, this feedback enables L&D teams to assess if there was a demonstrable change in the learner’s behavior.
Often, this is can be tricky. Although, learners had successfully cleared the assessment, yet there is no demonstrable change.
This may need re-assessment to determine why this is not happening.
- Sometimes, it could be because learners have no opportunity to demonstrate what they learned, and often, it may point to the need for reinforcement.
- There may be a need to have refresher programs to be offered over an extended period of time till the required gain is observed.
Level 4: Impact
Objective: At this level, the focus is to determine if the business saw the gain and if the required impact was created on account of the training.
From an evaluation perspective, this feedback enables L&D teams to review if the expected impact identified during the TNA phase indeed happened.
How Can You Use Kirkpatrick’s Model Of Training Evaluation To Measure ROI?
ROI determination (or Level 5): This is an add-on to the initial model (that has 4 levels) and is referred to as the Kirkpatrick-Phillips Evaluation Model of training.
In simple terms, you will have a positive ROI of eLearning if the demonstrable gain from the training exceeds the cost you incurred to create and deliver the training.
I hope this article gives you practical insights into how you can enhance the impact of each stage from TNA to the evaluation of its impact and see a positive ROI of eLearning.
If you have any specific queries, do contact me or leave a comment below.
Read More:
- Free eBook: How Can You Measure The Learning Effectiveness Of eLearning Courses And Create A Positive ROI
- How Can You Measure The Learning Effectiveness Of Online Courses And Create A Positive ROI?
- Case Study On How You Can Validate The Learnability Of Online Courses
- How To Predict And Measure The Learnability Of Online Courses
