Metrics That Actually Mean Something

A few weeks ago John Sullivan wrote an article citing a few disturbing recruiting numbers: 70% of participants are dissatisfied with the hiring process; 46% of new hires turned over within the first year (50% for new executives); and top producers produce 40-67% more than others. Sullivan recommended a variety of solutions. One of them included better assessment tools. A few weeks later, Lou Adler wrote an article suggesting that quality of hire was significantly more important than cost per hire. He also suggested a few ways to evaluate source quality based on candidate skills.

I applaud these comments. They have been a long time coming. But in many ways they are like advising Robert Reich to grow taller: easy to say, but doomed to disappoint. The formula for fixing these recruiting problems is threefold: 1) if you cannot define it, you cannot measure it; 2) if you cannot measure it, you cannot control it; and 3) if you cannot control it, you have a 50/50 chance of being wrong.

Defining Requirements

Most executives tend to measure performance by results. But wise ones know by the time results are posted, the activities that produced them are ancient history. Frustration ensues because even executives cannot control performance after the lights go out and everyone has gone home. It must be controlled in the moment.

Do you remember the old joke about the man who prays many years to win the lottery? Eventually, he hears God say, “Give me a little help here? Buy a ticket!”

Well, how does this sound? After years and years of praying for improved employee performance, God finally says, “Give me a little help here? Start accurately assessing important job skills!”

Buying a ticket or accurately assessing candidates are the only thing an employee, manager, or recruiter can control. The rest is out of our control. As a professional psychometrician (i.e., one trained to identify and accurately measure KSAs needed to perform a particular job), I can attest that nothing, I repeat, nothing has a better ROI than implementing an accurate hiring program.

But wait! There’s more! You also get candidates who think it is more professional; it’s exactly what the DOL recommends; every candidate is treated equally; turnover is reduced; individual performance rises; and, training decreases. Can anyone name other organizational program that can do all that?

No one expects you to be an overnight expert, but you can begin by using a basic rule of thumb. Divide job requirements into six general factors (i.e., mental ability required to learn and make decisions, organizational ability necessary to implement projects and plans, interpersonal skills necessary to deal with people, associated attitudes, interests, and motivations, special occupational knowledge required, and essential physical abilities). For each factor, identify the level affecting job success or job failure.

Clear and concise definitions are not optional.

Are you doing this already? A sure clue is relying on position descriptions, hiring managers who cannot agree on whether a candidate is qualified, or hiring managers who insist on seeing multiple candidates so they can compare them to each other instead of to the job.

Measuring Applicants

Picture this: A candidate applies for a job; he or she presents a resume, more than half of which, Sullivan says, contain lies; you have a few hours to ask questions about past job experience (which will probably be exaggerated); you may call a list of pre-screened references (who will probably lie to you); give a test borrowed from a training class (that has no proven link to job performance but people seem to like); get together with hiring managers to argue about whether a candidate is job-qualified (where everyone seems to have a different opinion); and, tentatively ask managers six months later whether they made a good decision to spend wisely tens of thousands of organizational dollars (expecting them to be honest). Sound familiar?

Is it any wonder why senior executives are always re-examining the value of HR? Why external recruiters have trouble differentiating their service from internal ones? Why organizations get sued for unfair employment practices?

What? You don’t use assessments? You only use interviews? You think the DOL guidelines are only for dummies? Keep reading.

Assessment is just a fancy term for measuring or evaluating job skills. Resume screens are assessments, as are application blanks, interviews, reference checks, tests, and photographs. Anything used to identify job-qualified applicants is an assessment. So, as Sullivan recommends, should recruiters use more assessments? Sorry, folks. That is like recommending fish swim more often. If you have asked a candidate a single question, you have used an assessment.

Now that you know you are an assessor, come to grips with the fact that gut feelings and unstructured interview assessments deliver reprehensible results. You don’t need more questions or interviewers. You need better tools.

Step on the Scale, Please

Hiring managers seldom have weeks, months, or years to observe and evaluate candidate skills … only a few minutes or hours. So, how does one get accurate and reliable data about whether a candidate has job skills? It helps to divide assessments into two classes: asking questions informally (i.e., person to person) or asking questions using a formal approach (i.e., pencil and paper or web-based format).

For people already familiar with structured interview technology, you can think of it as either asking the candidate to recount a situation, action, and result; or, formally controlling the situation, the possible range of actions, and using a standardized scoresheet to evaluate results. The objective in both cases is the same: evaluate whether candidate performance would lead to successful job performance.

An example of a poor assessment tool is a wife who asks her husband if she is getting fat. A foolish husband will size her up and render an opinion. A wise husband will instantly clutch his side, fall on the ground and scream, “Dial 911! I think my appendix has ruptured!” Measuring body weight based on opinion is irrelevant and dangerous. The same is true of assessments.

In choosing a set of hiring tools, always keep in mind that assessments that predict job performance are not the same as style or type assessments. Legitimate vendors are anxious to show documented proof that their assessment scores directly lead to employee performance. Illegitimate ones are not. Either way, the user, not the vendor, is solely responsible for assessment use.

Now let’s talk about the compelling power of human nature (or, why unstructured interviews persist). As pointed out by S.M. Colarelli and M. Thompson in the September 2008 issue of Industrial and Organizational Psychology, early humans lived in small bands of about 150 people, where physical survival depended on quickly sizing up someone based on face-to-face or word-of-mouth communications. People who made faulty decisions under these conditions often died before they could pass-on their gullibility gene. On the other hand, people who made better face-to-face survival decisions lived to have offspring who shared the same skills. Let’s acknowledge the force of nature.

Although most recruiters are not exposed to mortal situations, how often do you hear them say they want to get to know the candidate personally? Or a hiring manager say they know in their gut if it’s the right person? These are ancient survival techniques unconsciously doing their job. They have no place in recruiting. Once a hiring manager or recruiter decides the candidate is not Hannibal Lector considering whether to invite them to lunch, they should start accurately assessing skills leading to job performance.

Control

Controlling the hiring process does not mean asking managers to complete smile sheets or to rate results (results, if you remember, may or may not be a product of the “hows”). You need managers’ objective feedback about the new employee’s on-the-job hows so you can compare it to the data from your original assessments.

Start by using the six factors I recommend earlier. This model will provide a template for asking questions and comparing actual performance with measured performance. Look for things you might have missed, need to better evaluate, are duplicated, or unclear. Then make informed changes. Improving quality of hire is a TQM process applied to people skills.

Research has long shown choosing people with the right skills depends largely on clear definitions, measuring hows twice, using different assessment methods to measure the same hows, using multiple assessors, and measuring a full range of critical job skills. How do you get started? Well, you could hire a full-time psychometrician like the big organizations do, you might rent a expert for a few weeks to get started, or you might start using the system described above.

This completes the human performance cycle. If you cannot define it, you cannot measure it. If you cannot measure it, you cannot control it. If you cannot control it, you have a 50/50 chance of being wrong.

In other words, there is only one way to get there from here.