Let It Show! Let It Show! Let It Show!

“Oh the weather outside is frightful
But the fire is so delightful
And since we’ve no place to go
Let it snow, let it snow, let it snow”

Be ready or be snowed in.

Are you a person who loves the snow or one who dreads the snow? It often depends on how prepared you are for the upcoming winter and snowfall. If you have the right equipment, like a snow shovel, good tires, skis, snowshoes, or a snow machine (snowmobile), then snow can be an opportunity. You can plan on what you are going to do next. You can go outside, stay by the delightful fire, and even play.

We can also think of data as snowflakes. Each piece of data is unique. We can have a plan on how we are collecting, sorting, and analyzing data. Or, we can just ignore the data and let it just melt away, but it can limit what we are able to do. Data can be considered a worrisome threat or an opportunity.

Data as an Opportunity
Data helps us make a decision. When we look at a learning program or other initiative, we need to consider key decisions such as:
• Should we continue this program?
• Should we stop offering this program?
• Should we change this program?

We can then collect the right data to help us make a decision. To establish whether we should continue a program, we need to know the impact of the program on performance and business outcomes. We need to determine whether the performance of learners has changed, which impacts business outcomes like sales, retention, production, and customer satisfaction. We also need to make a prediction or hypothesis about the training. An example hypothesis is, People who complete this training will solve customer problems, which, in turn, will increase customer satisfaction. So, we count the number of problems solved, and then track the results of customer-satisfaction surveys. The data is an opportunity to determine impact and decide on whether to continue the learning program.

Data as a Threat
How is data a threat? Data can impact the way learning and development (L&D) organizations operate. Data, like course completions, hours spent in training, or number of courses offered, alone does not show an alignment between learning and the business. The data could show that people are finding other ways to learn such as through Google, coworkers, YouTube, and other informal learning offerings. The data could signal that L&D needs to move from building courses and eLearning to using existing content and job aids to enable learner performance on the job. The data could indicate that it is time to change the program or approach.

A measurement plan that answers the following questions should be in place to handle the data:
• What decisions do we need to make about a program or initiative? (Stop, continue, pivot.)
• What data do we need to make those decisions? (You need business and performance data.)
• Who can we partner with to collect the right data? (You need on-the-job data.)
• How do we sort and analyze the data? (Use your hypotheses to analyze the data.)

So, as the snow falls this winter, and as you consider the data that you collect, think about what decisions you need to make and what data you need to make those decisions. The more prepared you are, the more you can take advantage of the data and the snow.

About the Authors

Scott Weersing

What is learning analytics and why am I passionate about it? Way back when I was a newspaper photographer, I really wanted to know the who, what, when, where, and why about the story I was assigned to. I loved to find out more information so I could be in the right place at the right time in order to get the best photograph. The more information I had, along with personal experience, prepared me to take an impactful photograph. My journey to learning analytics follows the same path of asking questions and finding the right tools. When I started working in Learning and Development as an instructional designer, I always was curious about what the learners were going to do with the training on the job. Oftentimes, I would get a response from the SME that the new knowledge would just change behavior on the job. I guess I am a little cynical about the magic of training. Just wave the magic wand, attend the training, view the WBT, and your problems will be solved. I did not know the questions to ask to ensure that the training would be applied on the job, but my leaders noticed that I was curious and liked to ask questions. They asked me whether I would you like to be a performance consultant. After telling me what a performance consultant does, I said that it sounded great. Who wouldn’t want to solve business and performance problems with a series of interventions? It was my time as a performance consultant that I learned about the right questions to ask to get to outcomes and, in turn, I became fascinated with metrics. My favorite questions are still as follows: Can you tell me more about the problem? What have you have already tried to solve the problem? What would it look like after this problem is solved? What metrics or data do you have that show there is a problem? I became data driven to find the causes of problems and then track the solutions to see if we were moving the needle. The tools to find the root cause of a problem are the same tools to see whether the training is being applied on the job. I use interviews, focus groups, observations, checklists, and surveys to find out what is causing a problem, and then I use the same tools to find out what is happening after training and, in turn, making an impact on business outcomes. I would say that learning analytics and photography are similar in that you need to plan with the end in mind to collect the right information in order to tell a story and make an impact.