A two-part blog series 

--- 

The world has suddenly gone upside down and things as we know them are no longer necessarily relevant. The world of Face to Face training currently does not exist, and perhaps has changed forever. This has led to training professionals being agile and nimble, thinking of creative ways to deliver training virtually. A commendable job of it we are doing too!

But are we measuring the efficacy of these programs?

In this quick process, enough thought has not yet been given to evaluation of these online offerings. As the process matures, learning and development departments will want to know if the interventions they have designed are in fact helping the participants feel engaged, whether they feel it’s relevant or not and whether or not it is actually help them learn anything.

Shortly thereafter of course the business will step in as well and ask for business results of these learning intervention!

To help you along, we at C2C Organizational Development are pleased to present a two-part blog series on ‘Online Evaluation Best Practices’ to get you thinking, and act thereafter, about the impact of virtual learning sessions for you, your team and your business.

While we strongly recommend that you think about evaluation with the end in mind and think of business results first, we wanted to share some low hanging fruit with you quickly, so we are starting with this blog. The first in the series, this article focuses on the initial reactions and learning of the participants.

The Kirkpatrick Four Levels of Evaluation model, world’s foremost evaluation methodology, comprised of Level 1: Reaction; Level 2: Learning; Level 3: Behaviour; and Level 4: Results (illustrated below). My recommendations in this article are based on this model. You can find more information on this at the end of this article.

The magic mantra of training evaluation

The good thing is that in this changing world, some things remain stable and learning evaluation is one of them. The magic mantra for evaluation in the online world is the same as for any other kind of training. The mantra of Usefulness and Credibility:

Usefulness (effective training) - Information used to make decisions related to the program and progress afterwards

Credibility (training effectiveness) - Information showing your stakeholders that performance improved, and organizational results were positively impacted

So, what should we do?

Create simple but comprehensive evaluation forms

Particularly in these initial stages of virtually facilitated learning, when a lot of programs are being piloted in their virtual avatar, certain data will be useful to L&D professionals to help us enhance the efficacy of these programs. Some things you might want to consider are:

• The technology we use
• Efficient use of the different features of the technology
• Participants’ reactions to the offering in general
• How engaged they felt
• Duration of the intervention
• Whether they found the pre and post work doable
• Whether or not the facilitator was comfortable with the topic / technology
• The flow of the program
• The handouts sent to them

It is also advisable to use a Blended Evaluation Plan® approach, and include questions relating to the learning (Level 2) of the participants.

This list can go on, as everything may seem important and useful to collect, but the idea is to prioritize the ones most useful to create a comprehensive evaluation form. Here is an example of a form we created that fetched us some useful data:

You will notice that the open-ended question in the form above “What key concepts did you learn in the program? How will you apply them on the job?” gives you an indication of not only what the participants learnt in the session but also an indicative Level 3 (Behaviour) of how they will apply the learning on the job. This is a great way of collecting indicative data across different levels.

Try these ‘non-formal’ methods of evaluation online!

Here are some more tips on ‘non-formal’ methods of evaluation you can use to gauge your participants ‘reaction’ and ‘learning’ on online platforms.

Level 1 (Reaction):

• Use of the hand-raise, thumbs-up, thumbs-down icons at different stages
• Use the go-fast, go-slow options (in Zoom) to check if pace is comfortable for the participants
• Get them to put a quick word in the chat window about how they are feeling
• Use a quick poll to check engagement/relevance/satisfaction
• Keep an eye on participation in break-out rooms/chat windows to check engagement
• In Zoom and perhaps others, you can enable the option to check if the participants are keeping focus on Zoom or have toggled over to other apps during the session

Level 2 (Learning):

• Ask powerful debriefing questions
• Use whiteboard functions to get participants to ‘flip-chart’ discussions
• Conduct virtual role-plays and have other participants observe and give feedback
• Use the chat functions/unmute button to enable participants to answer questions
• Run polls to get yes/no answers to questions
• Listen in to discussions in break-out rooms to check understanding of concepts

Remember to use the mantra of Usefulness and Credibility and get the best possible data that will help enhance your programs!

Read Part 2 of this blog series on the business results aspects of evaluation – Level 3 (Behaviour) and Level 4 (Results): https://c2cod.com/blog/how-do-we-assess-the-big-picture-of-any-training-evaluation

So, what are you doing to evaluate your online learning offerings? Share your feedback and comments at preethi@c2cod.com