C2COD

How do we assess the big picture of any training evaluation?

Online Evaluation Best Practices – Part 2
Preethi B Rao, Head of Learning Effectiveness, C2C Organizational Development

Jun 10, 2020

In Part 2 of this two-part blog series, Preethi B. Rao, Head of Learning Effectiveness, C2C Organizational Development, turns the spotlight on behavior change and results of training in the virtual workplace.

---

Welcome back! In an effort to help you get some quick training evaluation wins, Part 1 of this blog series saw us going against the grain to get you started with tips on evaluating Reaction and Learning. Now, it is time to truly begin with the end in mind and think about evaluation from a big picture, results point of view.

Before I continue, a quick note for first-time readers of this series. My recommendations in this blog series are based on the Kirkpatrick Evaluation Model, the world’s foremost training evaluation methodology, comprised of Level 1: Reaction; Level 2: Learning; Level 3: Behavior; and Level 4: Results.

So, how do we assess the big picture of any training evaluation? It's easier than you think – and this time we bring some new magic mantras for you!


The end is the beginning


Learning interventions, particularly mission critical ones, should be started with a detailed consulting style discussion with the business leaders. Some questions to be put to the business leaders are:

  • What will success look like to you?
  • How will you know if the learning intervention has produced the desired results?
  • What business metrics will change if participants apply the right behavior? Get a range from the participant level metrics - lesser turn-around time - right up to business level metrics (increased customer satisfaction).
  • What will be the business repercussion of not doing anything at all?
  • Who will help support the participants through the learning intervention and beyond?
  • What specific behaviors do participants have to change on the job? Be as specific and granular here as possible.

Business Partnership is Crucial
Research by Prof. Robert Brinkerhoff in Telling Training’s Story shows that if minimal resources are employed during pre and post training, then the percentage of people who apply new behaviors is a dismal 15%.

The maximum need for business partnership is during the pre- and post- stages. Apart from providing pre-work to the participants, you could also think of pre-training preparation to include truly defining the problem, discussions with business to understand what behaviors need to change on the job, what success will look like etc.

The post stage is the most crucial of all, where the need for business partnership is most important. This is the stage where participants try new skills (and fail if they have no one to support them). This is where the managers/supervisors must support them by observing them, giving them feedback, coaching them, and encouraging and rewarding them. It is at this post stage that the learning really cements itself and begins to become behaviour on the job.

We as learning consultants must make sure to build strategic bridges with senior leadership and tactical bridges with managers and supervisors to ensure the business partnership. With great business partnership, we can change our results to –



Value must be created before it can be demonstrated

Just as a plant needs to be nurtured with sufficient care, water and sunlight before it grows to be a beautiful flower bush or a great fruit tree, a participant needs reinforcement, rewards, encouragement and monitoring to transform learning from the classroom to sustained behavior change on the job. This means that as part of the learning design, we must put together a great post training support package which includes all of these and leads to that behavior change.

Any program, however great, is not going to reap business results if behavior does not change. And hence, to demonstrate value to the business, we must create that value through nurturing the participants and providing them with the support they need.

A few things you can do to support participants may be –

  • Provide job-aid for them to use post-session
  • Set up observations and feedback session with their managers
  • Aid managers by providing them easy to use tools/templates for observation and feedback
  • Set up rewards systems/competitions for consistent application of changed behaviour
  • Set up frequent check ins with participants to check progress
  • Open discussion forums for participants to share success/questions
  • Set up strong buddy systems


Quick tips to enable training Behavior and Results in the virtual workplace

While none of the above three points are specific to online training, they are equally crucial in any learning environment that you need to evaluate. Here are a few quick tips for use at Levels 3 (Behaviour) and 4 (Results):

Level 3 Behaviour
  • Use platforms like Promote ( https://promoteint.com/) or others to enable more social learning.
  • The added benefit of using platforms is that managers can engage with participants in their own time and not necessarily in real-time. Research by Prof. Brinkerhoff has shown that even a simple ‘Like’ by a manager increases the engagement level of the participant to a good extent.
  • Set up manager-participant connect times to share progress, clarify doubts, feedback etc.
  • Share observations sheets for managers, with key behaviours to look out for when they are virtually interacting with the participants.
  • After an online session, give participants an assignment to complete on the job and ask them to report back on how they used the skills taught in the session during the assignment.
Level 4 Results
  • Ask participants indicative questions at the end of the learning sessions. For e.g. What initial success do you hope to see if you apply this learning?
  • Rely on self-reporting and get participants to report back on key metrics you have identified.
  • You can create a game-like environment around this if you are using a learning platform.
  • Maintain online editable shared dashboards that managers can easily fill in to track the agreed upon metrics.
  • Keep an eye on the metrics and provide more/less support as needed 
Share your feedback and comments at preethi@c2cod.com 

Preethi B. Rao heads Learning Effectiveness at C2C Organizational Development. She is a Kirkpatrick Certified Facilitator and one of a select few Kirkpatrick Gold Certified Professionals in India (and the world!). She has been working with Senior L&D professionals to help take their evaluation game to the next level and working with organizations to help streamline their evaluation processes.

References:

Register at https://www.kirkpatrickpartners.com/Resources for free online learning resources.