We want to pause and share some ideas and observations about how to tie learning to those impact metrics that you identified WAY at the beginning of your program.
And to start that conversation, we want to bring up some pretty shocking and provocative research from the Harvard Business Review. In a study of 1,500 business managers - the people who ran divisions and owned P&L statements - what percentage do think were dissatisfied with their L&D program?
Now - when we're live in a room and ask this question, things get tense. You can literally feel the air sucked out of the room.
Because unfortunately the answer is 75%.
And of course we immediately ask: why? If you go to this article and look at the original research, it boils down to that they can’t prove the program is working.
They can’t prove the ROI.
So they could have a great team, be happy in general, but if you’re a leader, not knowing the impact of learning is a concern.
Now as learning professionals we can have two reactions to this article. We can try to pick apart the research methods and write it off. Or we can address the elephant in the room and really take ROI seriously. And to our lessons very early in the program, NOT ROI in terms of learner satisfaction or completion rates, but hard costs and real business outcomes.
This is hard. And as someone whose led global learning programs that touched millions of learners annually, I know how many barriers there can be to measuring true ROI.
But I want to touch on some simple, smart first steps you can take to differentiate yourself as a learning leader and to create what we at Bright call un-deniable impact. You need to be maniacal about this topic, because if you are, you will eventually get to a place where you and your leaders know what you’re achieving. Maybe not overnight, but you’ll get there.
So let’s dig in - first, I want to suggest that launching Bright can help you on this front simply by providing an excuse. Almost an arbitrary line in the sand to compare BEFORE and AFTER.
In order to drive ROI we have to define it. And it’s more or less a metric that explains BEFORE and AFTER.
ROI is defined as:
The INCREMENTAL financial outcomes you achieve MINUS the investment you put in DIVIDED by the investment you put in during a particular time period.
Simple example:
I have an onboarding program that is 6 weeks long. We put about 1000 people through it per year. Each of the new hires make $20 / hour, and don’t hit the floor until after the 6 weeks. The cost of this program - even if we ignore any other technology or trainer or materials costs - is $4.8M per year. $20 per hour times 8 hours a day times 30 days times 1000 people… is a lot of money!
And I want to use this example because it centers on COST - which was that first metrics category we suggested in your Learning Strategy lessons. If we just assume that performance stays the SAME, if we can do ANYTHING to reduce the costs below $4.8M, then we can start to show ROI for our efforts.
So let’s say we can move this program to 5 weeks, and that our new hires on the other side are JUST as productive. Now the annual cost of that program is $4M.
The INCREMENTAL change is $800k.
Let’s just say for this example that we spent an extra $300k to make those changes. In this case, our ROI would be 166%. We recouped our $300k AND created additional savings beyond that.
This lesson is incredibly relevant for Learning departments. Whether it’s a day or a week or a month that you can get similar learning outcomes with fewer inputs, this shows that you’re thinking about the accountability of allocating scarce resources.
NOW - just to pause for a moment - I am NOT telling you to just remove training for the sake of it. I’m also not saying less training is always better. Remember that in this example, the performance of learners on the other side was AT LEAST the same. So if you can keep performance the same and reduce costs, you’re driving bottom line impact to the company.
Now -
Let’s keep going with our example. Because we assumed that performance was the same. In scenarios where it may literally be impossible to prove performance changes, starting with cost is a great idea.
BUT - the window if time during which you introduce new programs - whether Bright or otherwise - provides a window during which you can look for - in an exploratory fashion, jumps or changes to the steady state.
So as an example, let’s say this company we’re using in this ROI calculation is a customer support team. And that BEFORE your training program changes, those new hires took about 600 seconds on average to take a call. 10 minutes during their first month.
Let’s say that after your program you see they are getting those same calls done - with no increase in complains and similar customer satisfaction in 500 seconds. 8 and a half minutes.
In addition to the cost savings of your training changes, you now have performance improvements to layer in.
Without going too crazy on the math, I’ll tell you that in a customer support center, if you can
take calls all day at 500 seconds per call instead of 600 seconds, you can take like 12 more calls. Which is almost 2 hours wort. So each this worth like $40 a day per agent for every day we drive this improvement.
For our example, let’s stay thi bumps up the value of our training changes another $800k.
SO our new ROI is now:
$1.6M divided by the same $300k we invested. We’re now at 430% ROI. Most companies would call a $300k investment in learning ‘alot.’ But just like we saw in our early HBR article, that’s because they’re treating learning like a cost that they don’t know the impact of. And you can help change that.
Let’s do this same exercise, but for a moment I want to use a smaller, everyday example. It’s fun to run these kinds of numbers with GIANT budgets and programs. But if this same company - let’s use the same customer support group - has a sudden fire. The VP comes in and says ‘everyones making such and such a mistake’ We NEED training. can we put together a 30 minute class on this process or tool.
Sound familiar?
Ok - me too. Let’s think through the ROI of this and respond to the leader as a trusted business partner.
First of all, I’ll share there are 2000 people on the floor quote ‘making this mistake.’ If they’re making $20/hour, how much does it cost to put those people through 30 minutes of training?
Depends on a lot actually. Is it live in a classroom? Is it an eLearning? If it’s live it will definitely be more than 30 minutes. There’s time to get off the phone, go to the bathroom, go to the class, get back on the phone. And the admin time of scheduling it all.
But for simplicity sake, let’s say 2000 people off the floor for 30 minutes each = $20,000.
And how much is this ‘mistake’ costing us? If it were me, and I were trying to apply my Bright Academy lessons, I’d definitely ask this question. The answer could be ‘We don’t know.’ OR there could be a back of the envelope calculation from the operations team that says ‘every time we make this mistake we miss 3 calls. And we think that’s costing us about $25k a day.
Ok - COOL. I can work with that.
First of all, under this example, we’d get ROI if we do just about anything and reduce the mistakes. Because even 1 day of changing these mistakes ROIs.
$25k divided by $20k = 25% ROI.
For two days $50k divided by $20k = $150% ROI. You get the idea.
But what if your leader gives you like 10 of these projects at once? And you can’t get to them all?
Well, this way of thinking first of all gives you a way to prioritize those initiatives. You will earn tremendous respect from the leader if you can do some back of the envelope thinking with them about which projects will help the business most.
Second - and perhaps as important for you as a learning leader - is could you get the same ROI with less? So they demanded a 30 minute training off the floor. Well… would a 10 minute digital video achieve the same? Depends on the mistake people are making, but probably?
Or - is the mistake everyone? Can we pull a report and just train the people making this mistake? Instead of 2000 people getting trained, maybe it’s 500.
This is a hypothetical, but it’s an important exercise to go through here to help you really challenge the status quo and become a more valuable partner to your stakeholders.
Now - this is a lot of numbers, and it’s a very specific example. But I hope it’s sparking some creativity for you. Because THIS is an approach to thinking about training that you can apply almost anywhere. It can help you really lean into your designs in some places OR it can really help you step back from projects that may NEVER get the kind of ROI we’d hope.
The key criteria to run this kind of analysis tie into all the strategy topics we’ve already discussed:
- First - know your current state program costs. If you don’t you wont have all the info you need to run ROI analysis.
- Second - know your company or divisions current state BUSINESS performance metrics. Be aligned with your operations partners.
- Third - know your INCREMENTAL investment. This could be technology, FTE hours, coaching hours, wages spent on putting learners through training. Don’t work on projects without at least some sense of what you’re investing.
- And finally - have a control of some kind. In both the examples above we kept saying ‘assuming nothing else changes.’ You need some kind of control - a before/after or alternative scenario - to compare to. This could be the before/after of your release. This could be comparing learners in the program our outside the program. We KNOW there are factors outside your control like market changes, seasonality, technology changes and the like. But if you stop to think through those, you can still account for them when you explore ROI.
And your Bright Account Manager would be MORE than happy to brainstorm through this topic for your program with you.
Let’s move to a related topic. As you manage your immersive learning programs, the most likely first result you’ll see will be SLIGHT improvement. We definitely have customers who see big seven figure changes, but don’t underestimate the power of initial data. measuring ROI isn’t a once in a blue moon activity. If you get initial data back that says your ROI is low or moderate, GREAT! You’ve learned something. You can go to your leaders and say our changes helped, but not as much as we like. Here’s what we’ll do next. And then you change the simulations, or add more simulations, or require more iterations, or improve your AI rules. It’s a continuous improvement loop just like any other division in the company would use.
And let’s be real - I know that there is always going to be ‘the next big thing.’ No sooner have you launched one division in Bright than there’s another division. Or some new tech rollout. Or a process change. Or something is on fire.
I think THIS is arguably the number one root cause of that 75% number we opened up this lesson with. We can’t prove the impact of learning because we’re so spread thin and trying to boil the ocean with SO much training for SO many things. But this is counterproductive. We’ve talked in other lessons about the 80/20 rule, about respectful managing up and saying no to leaders to help them simplify requests or prioritize them.
But if you’re response to all of this is to roll up your sleeves and lead, then I am very very confident that you are well on your way to leading L&D into the future and establishing it as one of the most valued disciplines in your company.