The Vanity Metric of Course Completions or “Sprints vs. Marathons”

Get Updates via OttoLearn's Newsletter

Thank you!
Oops! Please try again.
The Cognitive Science Behind OttoLearn - OttoLearn Adaptive MicrolearningThe Cognitive Science Behind OttoLearn - OttoLearn Microlearning

Hi Josh,

About a week before I began getting my Ottolearn Mastery Moments, I had a popup window from Adobe appear on my screen as I was working on another project, prompting me to update my version of Flash. We do use Flash, so like an idiot, I clicked on the popup and asked it to start the update—and only then noticed that the url was not an adobe address. Of course, I closed the popup window using the X in the upper corner, which didn’t solve anything. Our IT guys did the best they could for me, but my computer is still compromised, and is being replaced.

Fast forward to Ottolearn and your Online Security for Employees course. After completing several mastery moments, I have now learned what to do with popups like that. This morning, as I restarted my computer again, that same Adobe popup appeared and this time I was ready! I opened task manager and killed that little $%^&^ dead in its tracks.

I know the point of letting us try out OttoLearn as participants was for us to experience the power of this platform from the learner’s point of view. I can tell you that I personally am very grateful for the training you provided to me, and the fact that I was able to let others in my company know how to kill off those nasty virus-carrying popups. Yes, it works. Yes, it’s fun! And yes, I have a true feeling of accomplishment.

I can’t wait for the point at which we can talk more about developing courses for our clients.

Thank you!

WHY We Built OttoLearn - OttoLearn Gamification

Put a fresh batch of course modules in front of your learners, and it’s a race to the finish line. The engaged learners bear down and get’er done so they can get on with other things. The less engaged jog along and eventually get their passing grade of 80%. Everyone’s a winner! They completed the race!

What really matters to you, Coach? Odds are that you're focused on a single metric - course completions - learners + modules x time = done.

Have you asked yourself if course completions are meaningful?

Course completions are so important to today’s eLearning fabric, that it’s one of only a handful of metrics that are even sent back to the LMS, along with duration and a grade.

It’s too easy to fixate on this easy-to-measure metric and assume that these course completion sprints actually mean something.

What if I told you course completions are (next to) meaningless?

Stay with me. There are essentially two reasons you built and deployed the course in the first place:

  1. Just to cover a liability, or
  2. To actually influence the behavior of your learner.

If you’re having them complete this course so you can have a record saying they’ve successfully run the race and this now reduces your liability, then you may still have an issue. Courts can look at corporate behavior and rule that even though learners completed a course, the organization didn’t actually enforce the behaviors and therefore are still liable.  

However, if you built the course to actually influence behavior, you have an even bigger issue. Sure, they’ve completed your course, but have they changed their behavior. Have they implemented your best practices? Are they going to improve any performance metrics? What is the long-term impact measured by behavioral change over time - the marathon that is performance improvement?

It’s only by identifying, measuring, and analyzing the actual metrics you care about that will demonstrate that  your training was worthwhile. The challenge is most organizations don’t have a systematic measurement of these KPIs and also only have the blunt instrument of a traditional LMS + SCORM courses to work with to deliver training.

For your learners to actually change behavior, it must be practiced, enforced, and reinforced. You need to keep best practices top of mind for your learners - you need to remind them, and ultimately, you need to reward them for moving the needle.

If you really want to change learner behavior over the long term, our OttoLearn platform can help you. Otto delivers daily drip training to your learners based on who they are, what they need to know, how deeply they need to know it, and for how long.

Let’s talk about how we can help your team run a better work performance marathon - not just win the course completion sprint.

Experience

Exp.

High

New accounting rules

Workplace violence & harassment prevention

Low

Framing a basement

Changing a tire

High

Low

Perceived Relevance

Option

Pros

Cons

Seats

  • Super easy to understand
  • Very predictable cost, if you have a specific number of users (eg: employees)
  • Doesn’t differentiate between users that have different volumes.
  • Have to purchase seats for your maximum number of users.‍

Active Users
(Typically the number of users that log in during a month)

  • You don’t need a license for every specific user, you can often only license half of your users (since perhaps only half ever log in during a month)
  • Typically there is a large cost for going over your licensed number of users, which can be incredibly expensive (eg: 5-10x more than your licensed cost)
  • You often have to “play games” as an administrator, not wanting to do a mass course enrollment if you have only have your users licensed in a month

For workplace Violence and Harassment Prevention, you have High Experience & Low Relevance.

  • ‍Why? You’ve worked in an office setting for years, so you’ve been through a number of workplace compliance courses.  You have a lot of experience in this area, and you really aren’t excited about it.  
  • What is going to motivate you?  Not much.  It won’t matter if there is a slick expensive video, you’re going to click through the material to get the quiz as quickly as possible, so make the material as short as possible or provide a pre-assessment to skip past the content.

For new Accounting Rules you have High Experience & High Relevance.

  • Why? This is directly related to your job, and builds on your existing knowledge.  You’re going to want to absorb the information efficiently, since you process related information every day.
  • What is going to motivate you?  Efficiently written information.  Just the facts, in as little a time as possible.  Video takes too much time, text is perfect.

For changing  a Tire, you have Low Experience & Low Relevance.  

  • Why? You don’t own a car or drive. You don’t feel that you need to know this, and you also know you can find the information on demand, if and when you actually need it.  
  • What is going to motivate you? Very little, this is the most difficult category. For learners in this category, it’s best to try to engage them through story telling techniques to create a sense of relevance. This could be through text and/or video.

For framing a Basement you have Low Experience & High Relevance.

  • Why?  You don’t know much about it, but you’re highly motivated to learn!  
  • What is going to motivate your? You’ll pretty much consume any form of training, but you’re most likely to seek out rich media experiences, to understand the topic from multiple angles. This is the only category for which video is a perfect fit.

Seats

Pros

  • Super easy to understand
  • Very predictable cost, if you have a specific number of users (eg: employees)

Cons

  • Doesn’t differentiate between users that have different volumes.
  • Have to purchase seats for your maximum number of users.

Active Users
(Typically the number of users that log in during a month)

Pros

  • You don’t need a license for every specific user, you can often only license half of your users (since perhaps only half ever log in during a month)

Cons

  • Typically there is a large cost for going over your licensed number of users, which can be incredibly expensive (eg: 5-10x more than your licensed cost)
  • You often have to “play games” as an administrator, not wanting to do a mass course enrollment if you have only have your users licensed in a month
No items found.

Engagement Factors

Experience

Exp.

High

  • Best possible quadrant for engagement 
  • Will overcome learning obstacles
  • Will find a way to learn, even if materials are poor  
  • Won’t need nudging or incentives
  • Text is great
  • Can easily learn something
  • May need to work up the energy to engage in low quality materials  
  • May procrastinate, so incentives can help motivate.
  • Text is great

Low

  • Wants to learn
  • Has little experience so can benefit from more instructional quality
  • Greatest benefit of video and other rich media
  • Worst possible quadrant  
  • May not have experience in the topic
  • May not really care about it
  • Will require a lot of motivation to see engagement
  • Video can help

High

Low

Perceived Relevance

Option

Pros

Cons

User-based
(Seats)

  • Cost predictability. Each seat costs you $x/month
  • Typically more expensive than a usage-based license

Usage-based

  • Typically less expensive than a seats license
  • Cost variability tempered by pre-purchasing usage credits that never expire and consume them over time

User-based (Seats)

Pros

  • Cost predictability. Each seat costs you $x/month

Cons

  • Typically more expensive than a usage-based license

Usage-based

Pros

  • Typically less expensive than a seats license

Cons

  • Cost variability tempered by pre-purchasing usage credits that never expire and consume them over time
Neovation Learning Solutions Team - OttoLearn Microlearning is a product of Neovation Learning Solutions

Want more posts like the one you just read? 

Subscribe to the OttoLearn blog mailing list and never miss a post!

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Extra! Extra! Read all about it:

Want even more? Check out the full blog archive.
/* */