U.S. government agencies are taking heed to the recommendations outlined in a 2009 ASTD report, The Value of Evaluation. In 2009, the American Society for Training & Development (ASTD) published The Value of Evaluation: Making Training Evaluations More Effective, a report that revealed how well training evaluation was meeting organizations’ business needs. Responses to the 26 questions led to disturbing conclusions, particularly that “Only about one-quarter of respondents… agreed that their organization got a solid ‘bang for the buck’ from its training evaluation efforts,” the report states.
From the ATD 2014 International Conference & EXPO: According to a 2009 ROI Institute study, the number one thing CEOs would most like to see from their learning and performance investments is evidence of Level 4 business results. Yet according to the
In this segment, Ken Phillips discusses using expert estimation and other techniques to isolate the impact of learning on business metrics while accounting for the influence of other factors.
In this segment, Ken Phillips discusses how to decide which learning programs you should evaluate for business results or return on investment and what metrics you should use to evaluate them.
The author presents an approach for using a rate to monitor projects, based on the re-evaluation of two key concepts, work total, and the rate per unit of work. By periodically recalculating and updating rate values, considering the comparison of the actual work vs. the planned work, the project manager may develop reliable databases for use in future evaluations and follow-up projects.
Replacing a team member can be a difficult and time-consuming process, from sifting through endless resumes to conducting interviews to on-boarding the new person. By effectively conducting formal reviews, supplanted by informal evaluations, project managers can address team members’ weaknesses, reward their good work, set future goals and implement an improvement plan, thus rendering the replacement of a team member less likely. This article explores ways to take the guesswork out of three evaluation conundrums when it comes to assessing team members’ performance. In doing so, it reports the results of a 2011 study–conducted by Harris Interactive–showing that organizations risk 250 percent of an employee’s salary in turnover costs because of poor performance management processes, including performance reviews. It then identifies three challenges that come up frequently during the review process and provides a solution for each challenge. Accompanying the article are two sidebars: The first sidebar lists three questions for every review; the second sidebar details the perfect type of review.
How to Write an Annotated Bibliography. An annotated bibliography provides summaries and evaluations of sources, while a traditional bibliography is just a list of citations for sources. As long as you keep this key difference in mind,…
The federal Race to the Top (RTT) program offered states millions of dollars to implement educational reforms that reflected federal priorities. Such priorities include building databases to store student performance data and implementing teacher evaluations linked to student performance on standardized tests. School administrators in 46 states are investing significant resources to overhaul their evaluation systems to increase the number of classroom observations and to put more emphasis on standardized test scores.
You’ve just put on a terrific training program. The evaluations are outstanding, participants’ comments are supportive, and scores on the post-test prove that a lot of learning took place. You feel great. You and your team believe you have scored a big win and your mission has been accomplished. The…
Every time you take the platform in front of an audience, your body language speaks loudly and clearly. Smile sheets or evaluations won’t help if you don’t know what your body is saying that your mouth is not. You must understand what you say without words to eliminate mental, physical, and emotional barriers between yours…
In today’s issue of The Washington Post, Federal Diary writer Joe Davidson calls out some content from the forthcoming issue of The Public Manager in his reporting about the Obama administration’s personnel efforts. The article “Obama personnel policies draw generally high marks” contains the following quote: “It’s about that time when performance evaluations of Barack Obama’s first year as boss-in-chief begin coming in. It helps when the evaluators are a nonpartisan group of experts who know something about the area on which they judge the president. Fortunately, that’s the case with several articles on Obama’s management agenda — written by a team of analysts, including industry and former government executives — that appear in the winter 2010 issue of the Public Manager, which will be available Friday at http://www.thepublicmanager.org. This quarterly journal is published by the (sic) Bureaucrat Inc., which describes itself as “a not-for-profit organization chartered and devoted to furthering knowledge and best practice at all levels of government. The authors don’t have a dog — or a donkey or an elephant — in the fight over Obama’s reputation. They’re Democrats and Republicans who push a good-government agenda.” The winter issue of The Public Manager will be released tomorrow, January 15. An Interview on Federal News Radio’s show “In Depth with Francis Rose” will also be tomorrow. For more information on The Public Manager, go to the website www.thepublicmanager.org.
Is your organization getting its value for money from its training evaluation? According to ASTD’s Value of Evaluation report only about one-quarter of respondents agreed that their organization got solid “bang for the buck” from its training evaluation efforts. With the tough economic demands, business leaders have to scrutinize costs even more, to find greater efficiencies. This highlights the need for the true value of evaluating learning to be realized and for current practices to adapt to increased efficiency and effectiveness demands. Ninety-two percent of respondents to ASTD’s Value of Evaluation report indicated that they measure at least Level 1 (reactions of participants) of the model. [more]However, use of the model drops off dramatically with each subsequent level, with only 18% measuring at Level 5 (return on investment). It therefore seems that organizations are evaluating at the first few levels and then dropping off completely. This opposes expert recommends of evaluating programs at all five levels, but trimming the number of programs that are evaluated as the level increases. It appears that the degree of usage of a Kirkpatrick/Phillips evaluation level does not tell us much about its perceived value. Although Level 1 is the most commonly used type of evaluation, it had the lowest rating of high or very high value. Only 36% of respondents whose companies use Level 1 evaluation said it had high or very high value. In comparison, Level 3 (evaluation of behavior) and Level 4 (evaluation of results) were seen to be the most valuable, with 75% of respondents indicating high or very high value for each level. Perhaps reconsidering what levels of Kirkpatrick/Phillips model are used for evaluation will benefit organizations in realizing the true effectiveness of their learning programs and increase their “bang for the buck”. Source: The Value of Evaluation: Making Training Evaluations More Effective (ASTD/i4cp) Click here to learn more about ASTD Research.
(From Indiana University) — The dreaded bell curve that has haunted generations of students with seemingly pre-ordained grades has also migrated into business as the standard for assessing employee performance. But it now turns out — revealed in an expansive, first-of-its-kind study — that individual performance unfolds not on a bell curve, but on a “power-law” distribution, with a few elite performers driving most output and an equally small group tied to damaging, unethical or criminal activity. This turns on its head nearly a half-century of plotting performance evaluations on a bell curve, or “normal distribution,” in which equal numbers of people fall on either side of the mean. Researchers from Indiana University’s Kelley School of Business predict that the findings could force a wholesale re-evaluation of every facet related to recruitment, retention and performance of individual workers, from pre-employment testing to leadership development. “How organizations hire, maintain and assess their workforce has been built on the idea of normality in performance, which we now know is, in many cases, a complete myth,” said author Herman Aguinis, professor of organizational behavior and human resources at Kelley. “If, as our results suggest, a small, elite group is responsible for most of a company’s output and success, then it’s critical to identify its members early and manage, train and compensate them differently from colleagues. This will require a fundamental shift in mindset and entirely new management tools.” Read more.
One event that triggers a dispersed series of similar events, that may yet again trigger even more events. While some reactions are linear, chain reactions can spread geometrically (aka snowballing), potentially causing large and unexpected impact. For example, one bank failing in the Great Depression would set off failures at many other banks. One currency can quickly devalue, setting off a chain reaction of similar devaluations. Atomic bombs are the result of chain reactions, with energy being released from a few molecules releasing the energy in its neighbors. A single match can burn down a forest. Chain reactions require some type of distributed energy patter. If one wants to stop a chain reaction, one can exhaust the energy in a controlled way. Some organizations go through chain reactions of key people leaving. Others get multiple subsequent bumps up or down in stock price. Reorganizations can cascade, as can new leadership and direction. Interest hikes, lowering or raising prices, spreading rumors, and viruses each have a chain reaction all of their own. Word of mouth about how good or bad a product, even formal learning program, is can spread like wildfire; when positive, it is called buzz or viral marketing, and can be helped along by a “tell a friend” button.
It is commonly referred to as Kirkpatrick Level 1. Did the students enjoy the program? 1) Quality advocates say that such feedback is important. Buzz marketers would say that the students are your best advocates, so you want them to be happy. It keeps the instructor on their toes. 2) There are plenty of people including Kirkpatrick who say that the information is fairly worthless. 3) Also, I have seen many situations where training groups use it because it is the easiest metric. It lets them off the hook from doing other evaluations. But I heard an argument yesterday from a 30 year veteran that had me thinking beyond the 2) and 3). 4) Does it put the students in the wrong mentality? Does it, as the instructor described yesterday, put the students in a mindset of learning back and saying, “OK, show my what you got? The lessons are your responsibility to teach, not mine to learn? Entertain me! Make it fun?” Does it position too much training as entertainment not training as responsibility to shareholders? What do you think?
(From The Huffington Post) — For better or worse, performance evaluations are a reality in business. But just because they are necessary, it doesn’t mean they are being done in a way that produces productive and constructive results. In fact, at their worst, they can cause employees to recoil, spreading insecurity, self-consciousness and fear. No matter what type of organization, performance evaluation goals should be fairly consistent across the board. Mainly, they should be used to communicate how well an employee’s performance meets the needs and demands of his or her role within the organization. So, yes, it is a performance management tool, but it is also a vital communication vehicle. If companies would see it as such, the process itself would improve markedly and net much better results. At its core, employees need to walk away from their evaluation understanding what effect their past behavior has had on the business and also, what they can do going forward to ensure they continue contributing to their own growth as well as to that of the company’s. Perhaps the most far reaching cause for problems during any type of performance management in general, and evaluation specifically, is the inherent discomfort and resistance that managers experience in having to deliver what they perceive as “bad news.” So, before further analysis can be devoted to what makes a review succeed or fail, one must first be clear about what organizational results this evaluative process needs to produce. Read more.
When showing a great simulation to an instructor, I often get back the following response. “It seems like students could game this.” Then the instructor leans back, smiling triumphantly, as if having delivered the killing blow. “Well, the simulation represents about 15 hours of student time. Sure, if they wanted to put in an additional 15 or 20 hours, they could probably get a better score not on their successful integration of productive knowledge, but on finding the cracks in the scoring algorithm and level design.” I reply. “But that would almost necessarily come after they learned quite a bit.” The instructor shakes his or her head. “The whole gaming thing troubles me. We need to have a higher level of integrity in any kind of grading or scoring.” Here, for the first time anywhere, is what I really want to say: ” Listen. I have been gaming classrooms for my entire life in order to get better evaluations, comments, grades, or certification scores. I have been dressing appropriately, feigning interest in topics that bore me beyond belief, cramming for tests in a way where my command of the information has a half life of hours desperately hoping that I forget the information moments after I write it down not moments before, skimming tangential sources to ask the one question that makes me seem much more knowledgeable than I really am, interviewing past students to see what will be on the test, playing back what the instructor said without understanding it at all, and pretending to take notes when I am really designing a biosphere in the margin. You want to talk about gaming? What do you think all of your students are doing all of the time?”
When should instructors fire students (ask students to no longer participate)? This is a more challenging issue in the corporate and government world, where training is more thought as a service, or a requirement, than the academic. This gets more interesting when simulations are introduced, and there is real work required from a student, not just showing up. Courses can also be several sessions, not just one. Some people view percentage who finish the course as a critical metric. And any pure e-learning course never automatically jettisons a student. But if students weakly approach a course, they go through the motions but don’t push, then that messes up any ROI and evaluations. The passive students also can become the biggest critics, resolving their own dissonance by lowering the view of the course instead of raising their own expectation for growth.
Executing Plans is a World Class Selling Core Competency and is an essential part of strategic planning. It allows you to focus on your priorities, assign responsibility, address current needs, communicate effectively, measure your progress and provides an opportunity to celebrate your success and make adjustments for future success. Follow these principle guidelines when executing sales action plans. Vision Vision should be a compelling, visual statement that is concise and empowering to your business goals. It should be read everyday and used as a constant reference point before making any decisions. Measurable Results Understand the two or three results that define true purpose of your business. They will be related to your vision, and will include profitability, performance, evaluation and customer value propositions. Key results Clear outcomes or differences in your business. Three to five of the key business processes that drive success. Focus will be on: marketing, selling, delivering, developing and managing. Be laser focused on what matters most and top priorities to achieve. Key Performance Indicators One or Two Measures and Evaluations showing quantitative business values that track targeted results. Improvement actions Projects, initiatives, investments and opportunities that help make the changes in your business. Celebrate your Success – Celebrate every step in your plan everyday. According to the book “World Class Selling – New Sales Competencies”, Executing Plans has several key actions: Definition: Organizes tasks and resources in a manner that coordinates resources effectively, maximizes productivity, and communicates expectations and results to stakeholders. Key Actions:
One should not evaluate a formal learning program to justify that the money and time were well spent. One should evaluate a formal learning program to get resources for the next program. If there is a “level 6” evaluation of a training program, it is, “did it lead to growth of the training program/growth of the training group/promotion of the sponsor?” If not, all other metrics are irrelevant. Of course lessons learned should make the next program (the X+1 program) better. But the driver of those improvements is to get an even better evaluation of X+1 than of X, so that selling the X+2 program is easier. There are those who view marketing as selective truth telling at best and lying at worst. That is too bad, because ultimately marketing is evolving the premise for a sustainable vendor relationship. It involves as much listening as talking. Credibility is hugely important, therefore. Honesty is a sine que non of any sustainable brand. But the culture of the group has to be growth through success, not just introspection. Evaluations should be done looking forward and outward, not backwards and inward.
People prefer to do business with people they like!How do you do that? One of the most rewarding aspects of great sales training is teaching others how building great relationships with prospects, customers and their client referrals. This article is about building relationships. Building relationships is foundational to performance improvement and transcends all areas of your life at work, play and at home. A good Trainer will teach that the most important person in a conversation is the other person! Steering and focusing the conversation on what is happening in the other persons life will be 90% of the conversation in the sales process. The other 10% will be the sales person questioning and gathering feedback. (Yet, statistics will show you that 80% of sales professionals still do not do this even after training because the skill is not practiced enough!) These statistics show that Sales Trainers and Talent Management still have a lot of collaborating to do! Sales Training Drivers.com is committed to helping the Workplace Learning Industry foster more of this collaboration and help sales professionals stay on target to meet their professional and personal goals. The business goal of building relationships is to teach how to move the sales forward for mutual benefit. Show your sales team that building rapport is broken down by value percentages representing the highest amount a prospect will likely receive and absorb your message during conversation. Rapport is comprised of your ability to use: 1. Words (7%) – 93% of people only listen to 7% of what you say and only remember 3%. 2. Tonality (38%) – the tone of your voice matched with someone else’s level of tone 3. Physiology (55%) body language, facial expression, posture, stance, composure, movements, gestures Learning how to use words and body language is crucial to successful selling (and training!). It must be done over practice sessions, one on one coaching, role plays, and measured evaluations. Practicing the use of specific words, tonality and physiology during the sales process is an art in itself. Less than 10% of sales professionals ever fully master it! Your ability to present yourself appropriately and ask questions will prompt people to give you the personal answers you need to solve their issues and sell them. Teach active listening skills and questioning techniques that check for agreement. It will show your sales team how to look at the prospects problem from their point of view. Have your sales team learn how to present your product or service as a valuable addition to the security, comfort, and enjoyment of your prospects life. Teach them how to become interested in their prospects lives. (family, job, and recreation activities, and financial concerns). This takes a lot of practice in building transferable behavioral skills in relationship building. We all want to feel special and we all want to feel OK. Many customers will go miles out of their way to do business with someone they like and will make them feel happy and appreciated. Once someone likes you, they will bring you into their inner circle of influence and their friends will become your friends. Your business will grow much faster while other sales people who do not build quality relationships are vulnerable to the ups and downs of the economy, trends and budgets.
I had a chance to chat with Rodrigo Corra Leite (@Rodrigo1000K) in Sao Paolo during my Brazil trip in July. Here is the a summary of our conversation around Social Media. 1. Tell me a little bit more about yourself and your company. Im Brazilian, 35, Professor and HR Manager. Today I’m responsible for the Corporate University in the 3Coraes Group, a company in the Brazilian coffee industry. 2. You are very active in social media, and have been using blogs and Twitter. Could you tell me how it is impacting your life? I always believe that it is so important to share the knowledge. I’ve learned that the more you share, the more you learn. I began using e-mail groups. After that I started using Orkut, that here in Brazil used to be very popular. I use the social media to see what is happening with the research and issues of authors that I appreciate, share my posts in my three blogs, look for human capital potentials for our business, and also to learn another language. I’d like to write more if I had time, since I enjoy receiving opinions, comments and contribuitions of my followers and friends. 3. Is your company using any social media (blog, Twitter, Facebook, etc.) at work? If so, how’s it working? Can you give me an example? Yes, in our company social media is present. We have the intranet, internal and external blogs. In the internals, the departments can write messages explaining their process, orientations, sharing articles about important changes, receiving questions, doubts or suggestions. The externals, we have one blog for each product of our portfolio. This way, we can interact with customers, to know needs and they can follow us on Facebook or Twitter, for example, to receive tips about how to prepare drinks, foods, and how to enjoy the experiences with our products. Here is one of our blogs http://www.mexidodeideias.com.br/ where our customers can look for recipes, share experiences and follow our experts. 4. Social Media is a big trend in the U.S. now, do you think it is going to be popular in Brazil as well? Why? Brazilian people like the interaction, our people are highly social. The social media now is present in our lives and jobs. For me that is like a great square where you can show your skills, products, to know people, to learn, to share, to collaborate. Today, recruitment, training, evaluations, are using social media. I believe that what goes around comes around. That is, if you collaborate to improve the knowledge of others, you get good chances to learn, to meet new friends like you, dear Wei. I’ve been a follower of ASTD for a long time on Twitter and I have been leaning a lot. I’ve been meeting people who believe that education can change the world and through knowledge we can make a difference. Thanks so much to this opportunity. See you around the social media space!
Wallace Hannum, a contributing editor to Educational Technology and a faculty member in the educational psychology program at the University of North Carolina at Chapel Hill, calls Ruth Colvin Clark’s latest book Evidence-Based Training Methods a “superb book” that “belongs in the hands of every training professional, not just on their bookshelves.” His review appears in the November-December 2010 issue of Educational Technology magazine, and it praises the book for moving from reliance on commonly held training myths to presenting training practices that are consistent with the empirical evidence about human learning. Hannum says that the “real message in this book is not the content it covers but rather how Clark presents this content by drawing directly on sound research into human learning. This is not just another book about training methods or how to develop training. Rather, Clark offers a fresh approach at the intersection of research and practice that is both based in empirical research evidence and completely practical.” In chapter 1 of her book, Clark debunks four training myths and provides four guidelines that will improve your training. (Some of them are probably going to shock you.) Here are some excerpts from the book: Myth 1: Learning Styles “The learning style myth leads to some very unproductive training approaches that are counter to modern evidence of what works. The time and energy spent perpetuating the various learning style myths can be more wisely invested in supporting individual differences that are proven to make a difference.” Guideline: Do not waste your training resources on any form of learning style-based efforts including instructor training, measurement of learning styles, or training methods that attempt to accommodate learning styles. Myth 2: Media Panaceas “When we plan instruction around the latest technology gismo, we ignore the psychology of human learning, which has severe limits. When we assume a technology-centric view, our focus is on all the wrong things. Instead of designing training to support human learning processes, we get caught up in the latest technology trends without regard for how they can be most effectively used.” Guideline: Ignore panaceas in the guise of technology solutions in favor of applying proven practices on best use of instructional modes and methods to all media you use to deliver training. Myth 3: The More They Like It, the More They Learn “[T]here is in fact a positive correlation between ratings and learning. But the correlation was very small! In fact, it was too small to have any practical value.” Guideline: Don’t rely on course evaluations as indicators of learning. Use valid tests to assess the pedagogical effectiveness of any learning environment. Myth 4: Stories (Games or You-Name-It) Promote Learning “The lack of universal effectiveness of most instructional techniques is the basis for what I call the ‘No Yellow Brick Road Effect.’ By that I mean that there are few best practices that will work for all learners and for all learning goals.” Guideline: Be skeptical about claims for the universal effectiveness of any instructional technique. Always ask, How is the technique defined? For whom is it useful? For what kinds of learning outcomes will it work? The rest of the book provides substantive information and practices that are grounded in research about a wide variety of topics, including If you want to learn more about the book and get a sample chapter, click here.
What is the most common method used to conduct Level 3 evaluations? According to the 2016 ATD research report Evaluating Learning: Getting to Measurements That Matter it is administering a participant survey, with 74 percent of organizations reporting using this method. Unfortunately, many of these surveys miss the mark because of poorly written questions, faulty survey formatting, and the use of measurement scales that potentially create bias in the data collected. In this session, you will…
In this certificate course developed by ROI Institute, build the skills needed to develop and deliver effective ROI evaluations for learning and performance, organization development, human resources, technology, change, and quality solutions.
If you need a low cost, quick to develop and deploy survey, then an electronic survey may be just the right solution. This issue offers practical advice to help you take advantage of this powerful tool. It demonstrates how to write and format effective e-survey questions, how to select the best method to disseminate your e-survey, and helps you avoid the common pitfalls of pilot tests and evaluations. This issue also addresses the technology challenges of implementing electronic surveys, as well as privacy concerns.
Did your participants learn anything from your class or program? That is the key question behind the simple, but useful smile sheet. This Infoline provides you with specific guidelines for performing level two evaluations, including a discussion of various test types, test formatting and design, and key administration issues. The issue includes a standard evaluation worksheet, a case study sidebar, and a helpful listing of online testing software tools. Author: Jack Phillips
Product SKU: 259814 ISBN: 978-1-56286-236-7
Pages: 16 pages Publisher: ASTD Press
E-learning has become a standard delivery choice for training professionals to consider when designing a performance intervention. Yet, how do you evaluate its effectiveness? This issue presents a four-phase model that will help you design an evaluation program to bring you the answers you need. The issue also explains the similarities and differences between e-learning and other learning evaluations, and provides clues to avoiding pitfalls.
All Categories Article Articles Books Courses Documents Job Aids Tools Magazines Podcasts Templates Top Collections Uncategorized Videos Webcasts Action Learning Advertising & PromotionsBusiness DevelopmentBusiness PlanningCareer DevelopmentCoaching CommunicationsConsultants Coordinating Cost Cutting Creativity and Innovation Crisis Management Customer Satisfaction Customer Service Decision Making DelegationEmployee PerformanceEntrepreneurshipEvaluations FacilitationFacilities Management Finances (For-Profit) Finances (Nonprofit) Fundraising (For-Profit) Fundraising (Nonprofit) Group Performance […]
Internet enabled technologies have impacted supply chain in major way. On procurement front, e-commerce has give birth to E Auction, Online Bidding and Global RFQs being floated with vendor evaluations being conducted through video conferring etc.
Many organizations are not effective in their approach to employee evaluations. Digital assessments offer many positive benefits to organizations looking to effectively evaluate their people
Whether you’re teaching, training or presenting, TurningPoint improves the success of your learners. The easy-to-use polling software provides enhanced tools to engage with your audience and identify their understanding. TurningPoint gathers detailed reports for meaningful, decision-making data important to you and your organization. Fulfill Your Every Need TurningPoint is a desktop application that provides a […]