Measuring the Effectiveness of Learning Programs: Do You Likert to Know?
No one likes to be deceived. Some people pride themselves on being able to root out lies and have built careers on doing so. Studies have shown that you can watch for cues in a person’s expression to discern emotional fact from fiction. Perhaps you’ve heard how a true smile is seen in the eyes. For example, if you know what to look for, you can tell when a smile is hiding a lie. There are “leaks” in someone’s attempt to deceive because their zygomatic major muscle (around the mouth) forces the expression; liars overcompensate with easy-to-manipulate muscles because the “smiling” muscles around the eyes are very difficult to fake.
When it comes to the world of workplace learning, deception is and has been rampant. And what’s worse, we’re lying to ourselves. The preferred vehicle of the lie has been through the use of a commonplace tool that is generically called a “smile sheet”. Essentially just surveys, “smile sheets” more often than not consist of a series of benign statements to which learners are asked to respond using a Likert-type scale.
Example: “Today’s training session will help me to do my job better.” – Strongly Agree – Agree – Neither Agree nor Disagree – Disagree – Strongly Disagree
Like the zygomatic major muscle, the smile sheet is as easy to manipulate as it is to distribute. However, beyond giving an indication of an instructor’s likeability or the general “niceness” of the trainees, a smile sheet measures very little.
There are three fallacies that propel the continued use of smile sheets. Fortunately, by taking a hard look at the mythical benefits of the smile sheet assessment, we gain a valuable perspective on how to measure learning effectiveness.
Myth #1: A singular training event can cause a long-lasting behavioral change.
In 1959, Peter Drucker coined the term “knowledge worker” to describe any members of the workforce who earn their pay via problem-solving through the deployment by convergence, divergence and creative thinking. A novel notion at the time, today it describes the vast majority of us.
Knowledge means knowing and knowing comes from learning.
Take a moment to imagine the bazillions of events that have been facilitated over the years to train workers so that they can learn and, subsequently, know. Essentially, the intention behind those training events was to make sure that “knowledge workers” become even more knowledgeable. So, just how effective were those events? Not very.
Today, thanks to multiple workplace learning studies, we are aware of the concept of the Forgetting Curve. It is, essentially, the mirror image of the Learning Curve. The takeaway of the research is that the steeper the positive slope of learning, the negative slope of forgetting will be nearly identical. Put another way, the singular training event, regardless of the instructor’s charm, the polish of the materials, or the elegance of the facility, will be largely forgotten soon after the event ends.
Smile sheets frequently ask participants to grade the instructor, the materials and facility. None of that matters if people are going to forget what they’ve learned.
To avoid the precipitous drop in learning that comes after a steep learning curve, learning experts now advise organizations to spread the learning over time. Termed the Spaced Learning Curve, effective learning programs are campaigns that include multiple, smaller learning “interventions”. Interventions can come in all sorts of shapes and sizes and deploying a variety of them is recommended to address the different ways people like to learn.
In summary (Myth #1), effective learning programs include a variety of interventions that are strategically “spaced out” over time.
Myth #2: The completion of a training event (or events) is an indication of learning.
Those who administrator learning management systems frequently generate fancy reports to show training completion. Filtered left, right and center, these reports, just like smile sheets, are made to impress. Comforted by numbers that show rates of training completion, organizational executives and learning leaders point to their high percentages and claim to have “trained the staff.”
Even if an organization has crafted a blended learning program and spaced out over time, the completion of training events (i.e. interventions) is only, at best, the bare minimum. Learning programs must include what are termed “Reactivation Interventions” that provide both cognitive as well as affective benefits. Cognitively, they put the subject matter back into the learner’s awareness. Games are great for this. Reactivation Interventions in the forms of audits, for example, are excellent ways to ensure that learners have an emotional relationship to the content rather than one that is simply intellectual.
In summary (Myth #2), ineffective learning programs end with completion reports. Effective programs are just getting started after learners complete training events.
Myth #3: Only formalized assessments are appropriate options when training on serious subjects.
Regardless of the industry, budgets for training staff are notoriously tight. Consequently, attention is given only to the subjects where people are going to learn how to either make the company more money or keep the company from losing money. Or, put another way, the serious stuff.
Conventional wisdom has been that you cannot train people on serious subjects in a playful manner. Furthermore, the assessments on such serious subjects must be formal; evaluations should be as serious as the subject matter.
The success of training interventions such as texting campaigns, polling, and viral videos have proven that informal learning is, at times, ideal when the aim is to get people to learn about serious subjects. Sharing rates, discussions, and general involvement rates serve as telling measurements of learning and effectiveness. This is largely due to the way informal learning interventions can more adequately meet learners in their current context. Effective programs, (again) spread out over time, can then guide learners toward more formalized assessments in a way that is less jarring to them. Programs that utilize gamification elements, for example, have proven to be particularly successful.
In summary (Myth #3), all subjects worthy of study are serious. However, effective programs avoid choking the life out of learning.
Daniel brings to VENZA over 15 years of providing innovative training and curricula solutions to clients in four countries (Norway, England, Brazil, and the US). He effectively works with clients by utilizing his creative spirit, international sensibilities, and knowledge of industry-wide best practices.
In addition to his expertise in training design and development, Daniel has served as a development director for non-profit organizations such as Estancia Jatoba, a reforestation initiative in Sao Paulo, Brazil. He also founded Music for Charities, a vehicle for charity fundraising through the promotion of independent musical artists. Daniel’s primary focus within VENZA is to facilitate internal operations so as to best serve the needs of our clients.