Let’s begin with the assertion that having the right job skills is at the top of the food chain.

There is nothing more dangerous to the bottom line than employees not being able to perform a job. In fact, without the mental horsepower, organization skills, and interpersonal skills to perform a specific job, an employee is a potential train-wreck (although a happy one, if that counts for anything) … and, yes, managers are considered employees.

The Manager

Research shows the greatest source of job satisfaction or dissatisfaction is a person’s manager. I’d like to pretend I had no troubles with managers, but in my own career, I have gone overnight from being a valued employee to receiving a one-way train-ticket out of Dodge: same person, same skills, same organization, but different manager. I have also moved in the opposite direction. The minute my manager changed, my train ticket was canceled and I was welcomed back into the club. Again: same person, same skills, same organization, but different manager.

I have also worked in and with different corporate cultures characterized by innovation and continuous improvement; where loyalty was valued more than ability; that resembled an institution for the emotionally dysfunctional; where time stood still; and, where feelings of support people were valued more than the product of the professionals. Yes, manager fit and cultural fit are alive and well affecting human performance everywhere.

One Man’s Paradise is Another’s Hell

Ben Schneider, chair of the I/O program at the University of Maryland, has written extensively about what happens when personal culture clashes with organizational culture. He calls it ASA — an acronym for attraction, selection, and attrition.

It sounds more complicated than it is. Applicants are attracted (A) to organizations based on their cultural reputation; organizations select (S) employees who seem to “fit” their culture; and, employees who don’t fit leave through either voluntary or involuntary attrition (A). ASA forces are like a corporate iceberg. It has enormous inertia to resist any change.

There are a variety of recognizable cultures. For example, innovative vs. traditional; interdependent team vs. individual; cooperative vs. competitive; arrogant vs. self-effacing; autonomous vs. controlling; and, trusting vs. defensive, just to name a few. Of course, these cultures also come in all combinations and permutations.

Making ASA Work for You

As I mentioned earlier, culture is a powerful force that is slow to change. Sometimes a change in executive management will have an effect; otherwise, changing culture is often like putting a frog in a pan: too hot for comfort and the frog immediately jumps out; but, slowly raise the temperature, and the frog adapts. (No actual frogs were harmed in this example.)

So, how do we match employees to a culture?

First, we have to recognize the three major forces at work in organizational culture: 1) direct manager, 2) job-related, and, 3) organizational. Because ASA works in occupations as well as organizations, we can generally assume job-specific managers and current job-specific employees already fit the culture or they would have left through attrition (of course there are always a few exceptions). This means the first step is making sure applicant first fits the motivational (aka cultural) requirements of his or her job.

After we know the motivational expectations of the job, we have to “overlay” the culture of the organization. For the most part, this means identifying factors that touch all positions. Most commonly, these include a preference for innovation, working in teams, being competitive, and not being narcissistic. You might think of the first three factors as being bright-side (observable, positive) and the last as dark-side (hidden, dysfunctional).

What do I mean when I say dark side? Bob Hogan, a preeminent researcher in the personality/motivation field (and also a subject-matter expert on my long-ago dissertation committee), has shown that bright side factors lead to upward career mobility, but dark side factors tend to emerge when employees gain position-power in the organization. Dark-side narcissists are usually extremely charismatic; however, inside they harbor deep-seated feelings of superiority and entitlement. One only needs to think of the many public and political figures making the news by shamelessly taking advantage of other people to further their own egotistical objectives.

The Process

Starting with a known performance framework, a professional test developer interviews people from the organization using that information to build a survey that includes both occupationally specific and organizational-specific factors. The developer gives the survey to a few hundred people. Their answers allow the developer to determine things like inter-item reliability and construct validity (this data enables the developer to make deletions and edits to ensure a robust test). If all things go as planned, the next step includes either a concurrent or predictive validity study.

Validity studies confirm the test actually measures what it was designed to measure. They involve one group of people taking the survey and another group of people rating them. A concurrent study uses employees already on the job. It is quicker, but since employees are generally alike (i.e., survived the ASA thingy), it is harder to find differences between them. A predictive study gives better results because new employees are more diverse than seasoned ones; but, it takes more time.

Conclusion

The fit between organization, manager, and employee is more complicated than most people imagine. Broad-scope, one-size-fits-all surveys tend to ignore critical job-fit factors. Good fit involves understanding the momentum and inertia of attraction, selection, and attrition and separating them into factors that affect both the employee and the organization. Done right, this kind of survey ensures getting the right people into jobs they will enjoy. Caution should be taken, however, to remember that culture usually tells us very little about job skills. Maximum performance requires measuring both.

The article cites a lady who specializes in what she calls energy profiling. She claims she or one of her licensees can examine your photograph to determine with perfect accuracy (her words) your personality type. Amazing! And to think all those psychologists who worked their way through graduate school, suffered peer-reviewed research, and spent tons of money pursuing advanced degrees for the last 100 years could have just looked at your photograph! Go figure.

I searched, but aside from watching an engaging streaming video taken in front of some very picturesque mountains, I found little proof that she was qualified to produce legitimate hiring tools. Her PR firm did claim she revolutionized the fashion and beauty industries by sharing her simple beauty/fashion assessments with women around the world; helped women align their physical features in perfect harmony with their clothing, jewelry, hair color and style; and provided pioneering insights on weight, sex & intimacy/relationships, depression, self-esteem, parenting, finances, physical health, and spiritual health. Wow. After all that, I guess hiring was the only field left to master.

I don’t know about you, but I like to see a writer have professional certifications or special education that would convince me they actually knew what they were talking about. You know, the same way we would expect a medical correspondent to actually have practiced medicine, a legal expert to graduate from an accredited law school, or an engineer to a have a legitimate engineering degree. But that’s just me.

She presents, as proof of her work, a collection of streaming video segments and personal testimonials from people claiming her system changed their lives for the better. Sorry, folks, this kind of “proof” is nothing more than personal opinion. If you want to know whether something is fact, you have to produce facts to support your opinion. Unbridled enthusiasm unsupported with expert knowledge is a dangerous thing.

I’m sure she is sincere about what she does. No one would make such wild claims unless they were. Unfortunately, using a photograph system to type people and predict job skills is a shining example of pure nonsense.

Let’s list a few facts prepared by the DOL, published in 1978.

As I claimed before, if you want to see an example of a rare event when the government got it right, read through the 1978 Uniform Guidelines on Employee Selection Procedures.

Clearly Define What You Want to Measure

The Guidelines suggest it’s a good idea to conduct a professional job analysis before starting any hiring project. A professional job analysis includes talking with employees to learn what they do, talking with managers to learn what’s important, and talking with people who know if the job will change in the future. Since these folks cannot be expected to know about testing, the analyst converts their information into measurable competencies and verifies it with a wide range of job content experts.

You know this step has been missed when people in the hiring chain argue among themselves or complain the recruiter keeps sending them the wrong people. A good job analysis reduces job confusion. BTW… I’ve never yet seen a professional analyst break down a job into this lady’s purported energy types. We must have all missed that class.

Step Two

Now that we know what to measure, it’s time to hit the books to choose trustworthy measurement tools. In some cases, this will be structured interview questions, pencil and paper tests, job simulations, realistic job previews, case studies, planning exercises, technical knowledge tests, and so forth. The important thing to remember is that any process used to separate qualified from unqualified applicants, even if it is a yardstick, is a test. I cannot repeat this enough: interviews, resume screens, application blanks, and even specialized recruiting sources are tests!

It would be nice to know the tests you used were accurate.

The DOL says you cannot rely on validity claims made by vendors, marketing literature, third party statements, or any other source. These claims probably have nothing to do with your job. Can you use a test developed for bankers to hire your banker? Only if you can show the two jobs are highly similar. That’s a good thing. Why spend tens of thousands of dollars based on false assurances?

Conclusion

Read up on the Guidelines. I’m sure this won’t be the last time someone will naively try to expand market share. But making a hiring decision based on a person’s photograph is not only bad science; it is completely irresponsible behavior

I can just see the future. “Position open for individual with a well developed root chakra, median energy navel chakra, and a mature third-eye chakra. Candidates with an overactive sacral chakra or an undeveloped heart chakra need not apply. Mature crown chakra’s always welcome.”

Does HR need any more trouble with gaining credibility?

In this article, I refer to my experience turning in a leased car. I always treat the companies I encounter anonymously; let’s just say this organization’s first name rhymes with “smells” and its last name rhymes with “cargo.”

Its logo, a cute little stagecoach pulled by a team of fast-moving horses, is so engaging that one can almost smell the sweat and manure. But enough about sweat. Let’s talk about manure.

When I asked the company representative where to return the car, the rep said I could not return it to a dealer (unlike every other leased car I owned). The rep explained they only had ONE (1!) drop-off location in Georgia. I would have to drive there … so much for convenience. On the other hand, the drop-off point could have been Guatemala.

A few days after I delivered the car to the inconvenient drop-off location, I received a bill for damage to the door. Since I had taken 360-degree pictures of the car before I turned it in, I knew the charge was nonsense. I called another customer-service rep in the End-of-Lease Department. After a short argument, they said they would look at the pictures I sent and call me back in 24 to 48 hours.

144 hours later

After 144 hours, I called again. This time I struck pay-dirt! An agent answered the phone in a droll, bored voice (I could tell I was interfering with his latte). I explained my situation and, although he sounded greatly inconvenienced, he grudgingly went to look at the pictures I emailed 144 hours earlier.

When he came back on the line, he said my pictures were inadequate. Another department would have to examine them (apparently, full-color digital photographs taken with a very expensive camera are insufficient proof when compared to personal opinion).

I said, “I was told that last week.”

He replied, and I quote, “Well, I’m telling you now.”

Stunned silence.

I said, “Can you please have your name?”

He was not intimidated. “M”, he drolled. (Again, I will be socially sensitive and not name names; suffice it to say M is the first name of a jar used to store fruits and vegetables).

I pressed onward. “And, what is the name of your supervisor?”

“We do not give out that information. You have to go through our process. You can’t just ask to talk to a supervisor.”

My mind raced: “Am I in a stooge in one of those reality TV shows?”, “Is Howie Mandel going to pop out of my phone?” … “Was ‘M’ in training for a government healthcare position?”, “Will I have to sacrifice a blemish-free goat before I can talk to a supervisor?”

Fast-Forward

I could go on about “M” and his smells-cargo employer, but the real point of this article is that hiring front-line customer service people is more important than most managers believe. Organizations are happy to get our money, but the true test of sincerity begins when things go wrong. Unless the organization is the only game in town, customer service is one of its few opportunities to show customers it cares — or not.

Clearly, M and his employer cared less.

A good customer service agent needs five critical skills. Some can be trained and some cannot. They need skills to: 1) empathize with the customer’s plight; 2) listen and gather relevant information; 3) use questions to clarify the problem; 4) engage in joint problem solving; 5) and, do follow-up.

Why These Five?

When a customer complains, he or she is experiencing two problems: 1) a task problem with the product or service; and 2) an emotional problem. Most people would agree they are not in the mood to solve problems when are angry or upset. This is why customer-service reps need empathizing skills. Empathizing helps the customer and representative relate to each other.

Only when the customer calms down can the representative begin to ask questions and gather information. This skill requires active listening skills and well-honed questions. Discovering what went wrong, and why, minimizes the potential for a repeat problem and maximizes the potential for a happy customer.

Joint problem-solving is the natural next step. It’s the time when customer and organization come together to work out a mutually acceptable solution. This does not mean giving away the store, nor does it mean the customer is entirely wrong (see the M-jar, smells-cargo example above). Finally, it takes some form of action in the form of follow-up or next-steps.

So what should recruiting look for in an applicant? What is trainable? Well, based on your experience, do you find it easier to hire someone without empathy and tell him or her to be empathetic, or hire someone with natural empathy skills? Is it easier to hire someone who is not smart enough, or hire someone intelligent enough to probe for information? Is it easier to hire someone with poor listening skills, or, hire someone with natural listening skills?

Back to the Ranch

From a customer’s perspective, M-Jar was in the wrong job. Instead of customer service, I would suggest he seek a career that used his natural ability to be snide and insolent … a career where he does not have to deal with intelligent life forms …

As for the company who hired M? Its hiring system either failed to identify all the critical factors important to performing a customer service job in a competitive environment, or it had no clue what to look for.

As for me, I’m out there sharing my story about a well-known company and using them as a personal example of what not to look for in a customer service job.

This is a fine strategy for making friends, but absolute nonsense for a recruiter (I had another word in mind, but it would have been politically incorrect.)

After some initial chit-chat, the only interview questions a recruiter or hiring manager should ask are ones that provide trustworthy and reliable data about whether the candidate has the skills for the job.

Canned Questions and Pickled Answers

Why do so many people miss the obvious? Anyone involved in hiring knows there is a big difference between acing an interview and acing a job. Furthermore, as we all know, working in a job we neither like nor have the skills for is a painful experience. I can understand this kind of clueless interview recommendation coming from an inexperienced hiring manager; but, should we accept this advice from someone who either passes himself off as an expert or who recruits for a living? After all, screening applicants based on job qualifications is the recruiter’s job, isn’t it?

What Do You Know and When Did You First Know It?

Asking the right interview questions requires knowing first what to look for. And, if recruiters have to resort to qualification questions like, “What is your greatest accomplishment?” or “How would you describe yourself?” or “What is your greatest strength or weakness?” it’s a sure sign they neither have a clue how to identify specific job skills nor how to measure them. Any determined applicant will rehearse answers to questions like these. Any experienced recruiter knows, at best, they serve as knock-outs.

I once worked with a “professional” recruiter who claimed he had a better way to ask interview questions. He drew close, looked around to see no one was listening, and whispered, “How would your best friend describe you?” (With great difficulty, I choked back a description I really wanted to use to describe him).)

I can just envision the flood of nasty-grams I am about to receive from recruiters in angry disagreement; but, I did not invent this stuff any more than Newton invented gravity. Best-practice interview techniques are supported by thousands of peer-reviewed investigations conducted by hundreds of experts. So, if anyone wants to argue, here is a list of roughly 200 universities. I’m sure they would love to hear your opinions!

Knowing What You Need

If knowing what to look for in an applicant seems so simple, why do so many people get it wrong? For one thing, it’s not as simple as knowing the results you want to achieve. Results do not tell you how a job was done — or even who did it — they are the scores at the end of the game. They do not tell you what the player did, when or why the player did it. Just knowing results leads to assumptions about the skills used to achieve them. You need more; otherwise your assumptions will lead to hiring mistakes. Let’s use Tiger Woods as an example.

Woods’ objective is to use the least amount of strokes to put a little white ball into 18 little holes. These holes are inconveniently located amid trees, sandy pits, hills, ponds, and grassy patches. The total number of strokes is the desired result; but, Woods is only partially in control. Between his first whack and last plop, Woods has to confront temperature, humidity, wind, clubs, lawn maintenance, equipment, other players, onlookers, physical conditions, and a host of other factors out of his control — any of which can affect his score. The same is true of job-holders.

Although we treat other people as if they are in total control of their performance, we reserve the right to make excuses for our own behavior. Psychologists call this fundamental attribution error. That is, you are totally responsible for whatever happens to you … but I am entitled to blame others for whatever happens to me. Attribution error interferes with hiring decisions every time we hear a candidate tell us he was unsuccessful. Fundamental attribution error addresses only one part of the human condition; halo is another.

Humans tend to use snippets of information to make sweeping assumptions about other abilities. This is called the halo/horns effect.

What? You misspelled the word disenfranchise? You must be a complete doddering idiot who needs help tying his shoes!

What? You reduced the overall consumption of paper clips in your last job? You are obviously qualified for our presidential suite!

How often have you heard someone suggest the first two minutes of an interview make or break a candidate? Do you honestly believe someone’s entire career-skill set can be measured in two minutes? The halo/horns effect causes us to make errors both for and against every candidate.

To summarize, there are many insidious forces actively at work whenever applicant, recruiter, or hiring manager meet: silly interview questions; fundamental attribution error; halo and horns; unclear expectations; and, assuming results and skills are related. It’s a mystery why more hiring decisions aren’t disasters!

Systems and Solutions

Think of job performance this way. Every employee has to confront certain kinds of situations. Generically, these situations require a combination of one or more of the following abilities: cognitive ability (e.g., mental horsepower); planning/organization; interpersonal skills; special skills/abilities; and, specific motivational components. It gets confusing when you try to evaluate more factors than these.

For example, if you have been trained in behavioral interviewing, you probably noticed that after four or five questions, you start getting the same answers. Or, if you asked a candidate separate questions about solving a difficult problem, making a tough decision or analyzing data, you begin to hear the same story. That’s because problem solving, decision making, learning, and analysis are often so entwined that it’s difficult if not impossible to separate them. Being impossible to separate means it is almost impossible to measure them individually. Instead, it’s better to look at them as a package called cognitive ability.

Ever hired a psychologist to administer tests to an applicant only to find the report awash in personality factors and character evaluations? Well, unless your psychologist has been trained in how to evaluate job skills, he or she can only do what they were trained to do: provide mental-health evaluations. Evaluating applicants’ mental health takes you right straight into conflict with the Americans with Disabilities Act. All you really want to know is whether the person has the skills for the job.

And another thing: avoid fuzzy concepts like business savvy, budgeting, tough mindedness, or drive to achieve. Fuzzy terms and hiring mistakes go together. One rule of thumb is if you cannot measure a job skill in a few minutes, then it probably is so complex that it cannot be accurately measured until the person is on the job a few months. Take leadership. Has anyone ever seen a “leadership”?

Leadership is the ability to bring a collection of individual skills together at the right place, the right time, and under the right conditions. More often than not, the skills vary with the situation. Sometimes they might require interpersonal ability, sometimes they might require analysis and correct decision making, and sometimes they might require planning. Leadership is not something you can see in a few minutes. It is a result of many things happening over time. Even the traditional leaderless group discussions that so many assessors are so fond of suffer from halo (e.g., extraverted people tend to perform better than introverts).

Interviews as Tests

It helps to understand that every problem has three components: 1) a stimulus; 2) an employee response; and, 3) a result. If you have tracked this article so far, you should understand that learning all three components are important to knowing whether the applicant has the job skills you need.

Vendors who sell behavioral interviewing programs often train participants on how to ask for background information, to probe specifically for what the candidate did or said, and to verify the results. These activities go by many acronyms (BEI, BBC, STAR, ABC, and so forth); however, regardless of the term used, the most important goal in behavior interviews is gathering sufficient information about all three components so applicant faking is minimized and specific applicant skills are clarified. Accuracy leads to better hiring decisions.

Simulations, pencil and paper tests, case studies, planning exercises, and the like, follow the same stimulus-response-result pattern. The main difference is you control the stimulus and know the result you expect. That improves accuracy.

Conclusion

The recruiting field is awash in nonsense and bad advice. This leads organizations to hire too many wrong people and reject too many right ones. Experts estimate this cost ranges anywhere between 20% to 50% of base salary. Being passionate about a hiring methodology and knowing it is valid and reliable are not the same thing. If a product or report seems off-target, ask to see studies proving scores actually predict job performance, look at the vendor’s professional credentials to see if they belong to the right associations (SIOP), or simply ask if the product was specifically developed to predict job performance. A vendor making claims that sound too good to be true are no different than the emails announcing your lottery winnings. A little common sense and education makes a world of difference.

Employee-productivity differences have attracted their share of researchers. Most agree that folks in the top half of workers out-produce the bottom half by about 2:1 (i.e., it makes no difference if people are shuffling papers or making widgets). And, when managers and knowledge workers are examined separately, the productivity ratio rises to 3:1, 4:1, or higher (i.e., responsible jobs have bigger ratios).

Productivity is more than a mental exercise. It shows up as absenteeism, errors, reduced throughput, turnover, low morale, rework, an excess number of employees, and so forth. Productivity losses are also sneaky because they are not easily seen; yet, they translate into hard cash: between 20% of base annual payroll leaked for unskilled workers to 50% for skilled and managerial employees — enough to separate a successful organization from a flop.

Converting payroll leakage into gross sales can be an even bigger eye-opener. Twenty percent leakage, for an organization that pays out 1/5 of its gross sales in salaries and benefits, would require a 500% sales increase to balance the books. Want to do more scary math? Calculate the incremental sales necessary to offset a 50% leak in managers and professional salaries!

Enter Financial Chaos and Uncertainty

We are in serious financial times. Opinions vary, but experts estimate our financial stress will last throughout 2009 and perhaps into 2010. The prosperity party is over. Like the dot-com bust, the world changed virtually overnight.

We cannot do much about external economic factors except dig in and wait. But, we can do something about employee productivity, especially when it comes to intelligent downsizing.

Ah’ll be Baack!

There are two ways to downsize. Most managers are accustomed to the Rambo model: plunge into the organization armed with rocket launchers, machine guns, and grenades terminating anyone in the line of fire. At the end of the rampage, the gross payroll body count is reduced; but, since both high- and low-producers are terminated without regard to skills, the organization continues to live with its 20% to 50% cash hemorrhage. Rambo-sizing is the norm.

What about examining employee performance before making termination decisions? Everyone knows performance recommendations are part fact and part fiction. Promotions and performance ratings are almost always based on personality and popularity — not specific skills. Just examine organizations that Rambo-sized their workforce in the past. What effect did it have other than forcing fewer people to spend more time at work? Termination decisions done without future planning are like bloodletting to rid the body of bad humours … they are more likely to kill than cure.

Planning Ahead

If management takes the time and HR is able to competently manage the solution, downsizing can actually help the organization get healthy and stay that way. It’s more like Mr. Spock than Rambo. It is rationally based. It begins by clearly defining the skills the company wants to leave in the past and acquire in the future. Here is an example.

We’reAllThatMatters is a legend unto itself. Employees generally want to work there because they can brag about the big-name. Unfortunately, people (read customers) outside the organization have a different opinion. Employees often treat customers rudely and without respect. For example, even if We’reAllThatMatters’ buggy bookkeeping system overcharges a customer 400%, employees treat anyone who complains as if it was his or her fault.

Now the organization must cut back its workforce due to economic conditions. Should it Rambo-size its employees? Should it ask managers for their subjective opinions about who stays and who goes? Should it amputate whole divisions? Since We’reAllThatMatters’ has been around some time, a majority of terminated employees may be over 40, raising the possibility of a nasty class-action suit. What to do?

Rambo-sizing would be a serious long-term mistake. The payroll would shrink, but both skilled and unskilled employees would suffer the same fate. Customer-sensitive as well as customer-insensitive employees would be terminated equally. We’reAllThatMatters’ payroll would shrink, but payroll hemorrhage would continue unabated. Logical-sizing would be different.

We’reAllThatMatters would a take hard look at itself and honestly calculate the financial impact of poor customer service on future business. It would then develop some key job profiles containing both technical competencies to do the job as well as customer service competencies it wants to build and retain. When this is complete, it would move on to the next step.

Employee-Level Evaluation

Individual employees would have his or her performance objectively evaluated using the list of necessary competencies as a target. For example, customer-centric skills might be evaluated by gathering past examples of service (e.g., similar to behavioral event interviewing), reviewing performance appraisals (to the extent they might include relevant information), giving tests, administering surveys, and so forth.

The secret to success would be to evaluate the skill set of every employee using an objective standard based on the organization’s tactical plan. Results for each employee would be anonymized and independently reviewed by a few highly competent managers. Employees who matched the profile would be retained, and those who did not would be reassigned or laid off.

Smart-sizing could be done with competencies such as analytical skills to develop better problem-solvers, initiative to encourage operational improvements, teamwork to develop better internal working relationships, creativity to foster new ideas and designs … the list goes on.

Final Question

However, there is a price to pay. HR has to develop the skills to help managers analyze and clarify the skills needed. It has to become proficient in accurately measuring competencies (real ones, not garden variety stuff), and it has to professionally manage the process. Managers have a price to pay too. They must have the patience to work through the details of smart-sizing, dedicate the energy and commitment to making sure the process is followed, and be able to clearly define the future at the employee level.

The outcome of this initiative is a smart-sized operation; in other words, the skills of the employees are intelligently aligned with the objectives of the organization. Overall, this should result in fewer employees doing more work (because each employee will be more skilled), less turnover (because employees will be more satisfied), fewer mistakes, better quality, and so forth.

The final question faced by everyone in the operation is whether saving 20% to 50% of base payroll is worth abandoning Rambo-sizing for smart-sizing.

(Although this article focused on the DISC, you could apply its comments to any test, assessment, interview, exercise, role-play, and so forth used to separate qualified from unqualified applicants.)

Some Background

DISC development began in the early 1900s when the Army asked psychologist William Marston to investigate why different soldiers who received the same training behaved differently. He published a report about 10 years later entitled “Emotions of Normal People.”

As far as we know, Marston’s objective was to describe “mental energy”… not assess and classify people for a job. Shortly afterward, another psychologist used Marston’s theory to develop a pencil-and-paper test. It asked people to choose between pairs of adjectives (i.e., which adjective is most like you and which adjective is least like you); then it added items together and reported scores for dominance, extraversion, need for security, and need for structure. (Remember that Marston was NOT trying to hire the most qualified people for a job — just explain normal behavior.)

So far, so good.

After that, the DISC grew in popularity and became another chapter in misused pseudoscience. Vendors and users alike tested everything making wild predictions about future performance. I have not verified it, but there were even some rumors about separate DISC profiles developed for inanimate objects, barnyard livestock, and certain vegetables. What happened? In short, as one great philosopher said, “When the only tool you know how to use is hammer, every problem looks like a nail.” That is, to the uninformed DISC user, every hiring problem looked like a DISC profile.

Test Construction

There are four things that define a good hiring test (i.e., one with scores you can use and trust):

1. It is well-constructed (e.g., all individual items supposed to measure expressiveness, actually measure expressiveness … and not dominance, security, or structure). This is termed inter-item reliability.
2. It delivers the same results over time (e.g., for the same person, November 2008 scores are the same as November 2007). This is termed test-retest reliability.
3. Scores predict performance (e.g., all high performers have similar profiles and all low performers have similar profiles). This is termed criterion validity.
4. It incorporates a theory of job performance (e.g., you are trying to predict job performance, right?). This is termed knowing your butt from your elbow.

Just common sense, right?

Overall, the idea of checkingoff descriptive adjectives seems like a good way to demonstrate preferences, but this design has some serious limitations. Consider, for example, the following items taken from one version of the DISC: Fussy, Self-reliant, Persistent, Optimistic, God-fearing, Devout, and Moderate. Can you clearly identify to which of the four factors they relate? Neither can I. Fuzzy items lead to fuzzy results.

In another case, a DISC vendor proudly published on the Web a study it claimed proved the VALIDITY of its DISC (i.e., if you recall, validity proves test scores actually correlate with job performance). Unfortunately, the vendor must have been disoriented because he only reported inter-item reliability — not validity — leaving the reader wondering if the vendor either successfully completed a class in psychometrics or read the Standards for Educational and Psychological Testing (both are requirements for test professionals).

Personality Scores and Job Performance

Are scores on personality tests highly predictive of performance? There is a long history of research showing: maybe. That is, if the employee has all the other performance elements necessary to do the job, and the personality test is job-related, personality can make a difference. Self-descriptive test scores represent how the applicant wants to present themselves — it may not be reality. So, even if an applicant tells an interviewer he/she is organized, it’s no guarantee he or she will be good with details. It is just bad science to claim the DISC (or any other personality test) will accurately predict managerial performance, capability for organization, character, or personal responsibility.

What about the four DISC factors? Remember that Marston never intended the DISC be used for employee selection? Decades of personality research shows only factors consistently relate to job performance: being extraverted, not being neurotic, and being conscientious. Skills-based research, on the other hand, shows that intelligence is the greatest predictor of ability (e.g., people who are job-smart outperform those who are not). I don’t know about you, but it seems the DISC falls short on both. But, if you have been using the DISC as a hiring tool for some time, and rigorously monitor your results, you probably already know that.

What about the most-least scoring system? Well, that leaves something to be desired too. You see, if there are eight expressive adjectives and I check-off four … but you check-off a different set of four expressive adjectives … have we taken the same test?

The same goes for comparing expressive adjectives with ones for dominance, security, and structure. Ipsative scoring (as it is termed) tells us a great deal about individual preferences, but it is generally discouraged for hiring by test professionals.

So What Good Is the DISC?

If you need some help understanding broad differences between people, and you are willing to take a test never intended for serious business use, then the DISC is a fun tool. If you understand that when you describe yourself as primarily dominant (or any other factor), your scores will probably show you are dominant (amazing!); or, when you want to facilitate a quickie workshop in communication styles, the DISC is for you.

Always remember, the vast majority of personality tests were designed to evaluate differences between people — not evaluate people for jobs.

Wrong-Way Thinking

There is a common fallacy among a significant number of people that anyone can do anything: a good-looking applicant will make a high performing employee; a high performing employee will make a good manager; or, a highly skilled employee in Job A will also be a highly skilled employee in Job B.

Sorry, folks. We all know from experience this is general nonsense. Stories are legend about a top salesperson or technical whiz who failed as a manager; or, about a marketing whiz-kid who fast-tracked into the executive suite only to crash and burn on the job.

Let’s put this puppy to bed. The only time that past performance in Job A accurately predicts future performance in Job B is when both jobs are require virtually the same competencies. If Job B is different, requires more competencies or better quality ones, all bets are off. In fact, the only reliable way someone might even guess at future performance is to know the employee screwed up his or her last job.

Consider the Peter Principle. If you don’t know the term, either Google “Peter Principle” or look it up here. In short, Dr. Laurence Peter gave multiple examples of how employees tend to rise in the organization until they reach their natural level of incompetence. His message: every time that job requirements change — or an employee changes jobs — there is a strong probability that they will not be competent in the new role. Although Peter uses corporate ladder-climbing as his examples, his principles apply equally to all people holding jobs. The Peter Principle is a classic must-read for every recruiter or hiring manager.

In the next few paragraphs, I’ll explain why the Peter-Principle is alive and well.

Little Observations = Big Assumptions

The world is a huge, complicated, and unpredictable place. If we had to investigate thoroughly every situation before making a decision, we would run out of hours in the day. So, evolution has blessed/cursed us with an unconscious tendency to make snap decisions based on something we learned earlier. For example, we consider taller people to be more skilled than shorter people (e.g., adults are always bigger than kids); we assume that best-dressed applicants will be better performers than other ones (e.g., we equate attractiveness with skill); or, we assume bad job experiences were the applicant’s fault (e.g., blame the victim).

We call this the halo/horns effect: or, use a snippet of data to form an overall opinion (i.e., a spelling mistake must mean incompetence; a charismatic employee is also a competent one; or, graduating from an Ivy League college is better than a public school).

Little observations often lead to big mistakes.

The Interview Hammer and the Applicant Nail

Ask any recruiter manager who relies on (unstructured) interviews and he or she is likely to swear by their accuracy. Look at any sales manager and he or she will say they are a good judge of character. However, when you look at the employees hired by the same person, you will wonder, “What was this guy/gal smokin’ when they hired the troglodites?

There is a major disconnect between being asking get-to-know-you questions and evaluating job skills; industry-wide research shows it to be about 50%. That is, interviews may screen out blatantly unqualified applicants, but it’s a coin-toss whether survivors have sufficient job skills. Ask yourself, did the person who interviewed/hired you really know whether you could do the job or not?

Any recruiter or manager who is only familiar with interviews (or a silly training test), is doomed to believe to think one tool can measure every job skill.

Job Funnels

Jobs are more than titles. They are more like an upside-down funnel. As a general rule, higher-level jobs need more and better competencies. Take sales, for example. A true professional salesperson has exceptional rapport building skills, is skilled asking the right questions at the right time, only makes presentations that fit the prospect’s needs, and can assist buyers to overcome the fear of making a bad decision.

Now, move the salesperson into a management role. The job requirements change significantly. In addition to individual sales skills, the person must become a coach, a planner, and a sales diagnostician. Skills that came naturally as a salesperson must now be broken down into hundreds of teachable elements. In addition to having the right skills, the new manager must be as excited with the thrill of the coach as well as the thrill of the close.

The following job roles point-out a few of these differences:

1. Individual contributors must have skills to do the job without assistance;
2. In addition to their individual contributor skills, first line managers must also be skilled at coaching, planning, and diagnosing subordinate shortcomings;
3. Mid-managers usually require skills in group influence, tactics, analysis, planning, and mentoring; and,
4. Executive managers are usually required to be strategists, navigators, and motivators.

We can’t always rely on job titles to describe job functions. I have seen occasions where a fancy title disguises an individual contributor, as well as complicated jobs with simple titles. The key to successful performance is to know what exactly what skills are required; then, use a variety of structured interviews, pencil and paper tests, and simulations that accurately evaluate them.

Imagine advertising for superheroes. There are a dozen steroid-pumped, ego-centric applicants sitting in your waiting room wearing masks, capes, and tights. Each hero claims to have saved the world at one time or another. You hire three of them. Six months later, how do you evaluate your quality of hire?

Evaluating quality of hire requires looking at performance in a different way. It requires mentally separating the “how” from “what.” The “how” represents what the superhero says or does and “what” represents the outcome. Here is the hard part to accept: evaluating quality of hire depends almost entirely on evaluating “how” the hero performed the job, not the outcome. Regardless of opinions to the contrary, “how” is the only part of the job under the hero’s control. It is the only thing separating one hero from another.

Here is an example that may explain this idea.

It’s a particularly bad time on earth. Asteroid showers are occurring periodically, keeping the super-heroes busy. When he was on duty, Clock-Man reacted by turning back time. On her shift, Wonder Woman pulled the asteroids into new orbits with her lasso. And when it was his turn, Superman flew faster than a speeding bullet, smashing them into smithereens. The “what” was the same for all three: reversing time. Using lassos and brute force were all examples of “how.”

What about evaluating the quality of hire? Let’s look closer.

Clock-Man’s action trapped the world into a year-long time loop. The world kept rewinding and playing back. The asteroids never hit, but the world missed celebrating Thanksgiving and Christmas, causing turkey famers all over the land to go bankrupt. Superman decided to stop first at Starbucks. When he finally finished, the asteroids were too close to be completely destroyed and one piece broke off, annihilating New Jersey (starting a lively argument about whether New Jersey was a fair trade for a decent Venti Mocha Latte).

Wonder Woman was shopping for glassware and misplaced her invisible plane among the fine crystal. When she finally found it, was able to lasso the largest asteroid, but a small one evaporated Paris Hilton and a small platoon of paparazzi (although no one except Mr. and Mrs. Hilton seemed to care). Now can you evaluate quality of hire?

Evaluating quality of hire is based on the same elements as determining which applicant to hire…you have to decide beforehand “how” a job to be done. Looking only at results can confuse performance because there are so many other things that can affect them. Superman might have been successful if he was motivated. Clock-Man should have thought through the long-term consequences of time-tinkering. Wonder Woman might have been more successful if she would have recognized the problems associated with finding an invisible glass plane in a clear-glass factory.

Human performance always has three components: 1) an antecedent or event; 2) the candidates’ response or behavior; and, 3) the consequence or result. Folks call these the A-B-C of performance. The antecedent and consequence are the “whats” (i.e., the results). The candidate’s behavior is the “how” (i.e., what the employee said or did when confronted with the situation). “How” is what we use to define job requirements, select and promote employees, and evaluate quality of hire.

You can think of every job as having standards for motivation, organization, analytical thought, learning, and behaving. For sales jobs these might include competitive drive, time and territory management, sales development strategies, learning new products, making presentations, and so forth. Management jobs might include the motivation to direct and develop subordinates (instead of doing it yourself), achieving objectives, solving problems, managing the marketplace, and coaching skills. Jobs always have cognitive components and behavioral components.

The key to understanding “hows” is knowing which behaviors vary with the job holder, which are necessary for successful job performance and which are associated with failure.

Why don’t we more often use “hows” to evaluate employee and applicant quality?

1. “Hows” often occur hours, days, weeks, or even months before we see results. We either forget or overlook them.
2. Results are usually in-your-face singular events that command attention, whereas “hows” are more subtle and might occur together in clusters.
3. We look at results and jump to conclusions about “hows”, often taking them for granted.
4. Some people take credit for other’s “hows.”

I’m sure you can think of more, but people seem to have an intuitive understanding that how’s are important. But because most folks are not measurement experts, they get them confused. For example, let’s look at a sample list of recommended action verbs (i.e., hows) taken from a career-services center. They recommend using words like administered, analyzed, attained, chaired, contracted, consolidated, coordinated, developed, and strengthened. Unfortunately, if you are screening resumes or interviewing applicants, the only verb that even closely resembles a candidate-centered how is “analyzed” …the rest invite assumptions.

People are wired internally to make fast decisions based on little data. While this might be a good survival strategy, it leads recruiters and hiring managers to make huge assumptions about candidate skills. Negative information, for example, (e.g., a typo in a resume) leads us to assume the candidate is sloppy and inept. Positive information (e.g., high sales dollars) leads us to assume the candidate is highly skilled. A successful recruiter who knows how to identify and evaluate candidate “hows” will both recruit better candidates and be able to better evaluate quality of hire.

The phone rings. Someone on the other end says he or she wants to build (or buy) a Web-enabled hiring test. Let’s say it will be for salespeople (generally the caller is a recruiter or HR manager, but sometimes he or she is a gopher).

After discussing the idea for a few minutes, I make a few suggestions. These always include following the ‘Guidelines’ to make sure the test is based on job requirements and business necessity and following the ‘Standards’ to make sure the test actually predicts job performance.

In almost every case, the caller is aghast at the work that needs to be done.

“All I want is a test!” they say.

“You want one that works?” I ask.

“Yes. But that’s hard!” they say.

“What’s your point?” I say.

On the other end of the phone, my keen bat senses hear muttering about me being a “jerk,” then dialing someone who will sell them the “mother of all tests”?one the vendor promises will work, regardless.

Why All the Fuss? A Test is Just a Test, Right?

The Guidelines and the Standards are not “nice to know” (i.e., limited to eggheads, legal eagles, and companies with U.S. operations). They describe how to define and evaluate job skills. That is, they first recommend test users define critical elements of the job based on job requirements and business necessity; then, they describe three ways to make sure test scores accurately predict performance (e.g., criterion, construct, and content validation).

Let’s reiterate. Step 1: Define job requirements and business necessity. Step 2: Make sure the test is predictive and stable.

Clear definition and evaluation is good for the hiring organization and good for the applicant. This principle works in all cultures and countries. So, if you plan to use a Web test, it’s a good idea to know the test actually separates qualified applicants from unqualified ones.

If you don’t get anything else, patch this into your screensaver: the only people who think it is too much work to follow best practices are people who don’t know how to do it.

But, no harm is done, right? Wrong. Highly effective hiring tests that claim they have no adverse impact or have been “validated” by the U.S. EEOC are as legitimate as the email announcing you won the lottery in Botswana.

Bad tests are really bad news for employer and applicant alike. A bad product backed by good-sounding marketing claims is still a bad product. And whether the user is in the U.S. or not, the test consumer, not the vendor, lives with the consequences of test use!

So, even if the vendor claimed his test was validated to grow hair on bald applicants, transform ugly employees into movie stars and cure morning breath, it would be your problem, not the vendor’s, to prove it.

Cause and Effect

There is a good reason why sailors advise passengers not to spit into the wind. The same is true for feces, fans, and bad tests. Eventually, even clueless test purchasers learn a weak test does not work as promised. You see a test that is not based on job requirements and business necessity, nor validated for the specific job, is designed to pass too many wrong applicants and fail too many right ones. It will show up on the job. That’s why the Guidelines and Standards are so valuable: they define exactly how to identify, qualify, and use a test that contains the least amount of error.

The bottom line is no matter how many years a person has been a recruiter; no matter how smooth his or her marketing campaign; no matter how certain he or she is about being a recruiting expert; and no matter how famous their organization, the ‘Guidelines’ and ‘Standards’ set the bar for measuring job skills.

Let’s examine how the ‘Guidelines’ and ‘Standards’ work for a sales position.

Sales Hiring 101

First, any method of separating qualified from unqualified applicants is a test. And “assessment” is just another word for “test.” We assess resumes, application forms, and applicant skills. The vast majority of organizations, unfortunately, use a two-step assessment process. Step one: use an interview to screen out most of the riff-raff. Step two: let the job screen out the rest. The two-step process explains in large part why 20% of salespeople generally produce 80% of the sales. Only riff-raff were screened out pre-hire.

Screening out riff-raff is easy. All you have to do is get to know the applicant, examine earnings statements, and dislike his or her personality. Normally, organizations screen-out 3.5 applicants to get one promising employee. On-the-job performance screens another one of two. Over time, this makes the final hiring ratio about 7 to 1. Riff-raffing is the norm and riff-raffing is expensive.

Let’s look at the cost of using the job as an assessment in terms of training, travel expenses, management coaching, and salary for six months. We’ll be conservative. One week training = $2,500; sales travel expenses = $100/day for six months or $12,000; coaching time = 15% of manager’s time or about $6,000; and, six months’ salary and benefits = about $36,000. This totals about $56,500 per salesperson (ignoring recruiting fees, lost customers, empty territories, and so forth). Bottom line? In round numbers, riff-raff assessment costs upwards of $50,000 for each lost salesperson.

Error-Free Hiring?

Mistake-free hiring is pure fiction, but doing a better job screening is not. First, you have to fully understand your specific sales job and the critical skills that separate the successful from the unsuccessful salesperson. This kind of information is seldom obvious. It does not come from generic tests, averaged scores, and calculating group norms. Generic norming is bad science. It serves as an example of wrong-headed test practices.

A trustworthy and reliable test involves in-depth understanding of critical job functions, measuring every critical skill area at least twice, doing a formal study to confirm scores predict job performance, and monitoring adverse impact. In professional terms, this is called job analysis, validation, multi-trait-multi-method assessment, adverse impact monitoring, and continuous improvement. If it sounds like a good way to do business, it is. If it also sounds like hard work, it is.

In the next few paragraphs I’ll briefly describe what to look for in a sales selection system.

Professional Job Analysis

As mentioned above, a professional job analysis does not consist of giving everyone a questionnaire and comparing top-performer scores to bottom performers. This is the first sign of buyer-beware because it makes some huge and often wrong-headed assumptions.

It assumes an equal playing field. That is, all productivity results are equivalent. New accounts, customer service, market conditions, and expanded accounts are all rolled-up into the same category: productivity. In some cases, overall performance might even be complicated by (gasp!) skillful manipulation of numbers. Separating salespeople into top and bottom producers based on sales dollars is a sure clue the analyst does not understand sales.

Suppose you are like most folks in the hiring business and you expect your test to accurately predict job performance before you commit big bucks to salary. By definition, your test should measure something that causes performance. If you give one big test to everyone without knowing explicitly what you want to evaluate, you fall into the “correlation or causation” trap. As an example, ice cream sales and shark attacks have a strong positive correlation. Does that mean shark sightings cause people to eat more gelato? That Ben and Jerry’s Chunky Monkey is a poor shark repellent? Or perhaps sharks have a seasonal business they don’t want people to know about? Homegrown questionnaires often confuse correlation with causation. Just remember: Unless water-born ice-cream is proven to attract sharks, one does not cause the other.

A good job analyst knows how to identify key skills that make the difference between successful and unsuccessful cold calling, repeat sales, strategic selling plans, customer service, and so forth. In many cases, they may involve totally opposite skills. Treating sales production as a discreet measurement point is like putting fruit salad in a blender, pressing the annihilate button, and testing the puree for peaches. A professional job analyst knows key information can only come from people doing the job, not from supervisors or aggregated production data.

Let’s say the analyst has done his or her homework. Now what? The hiring manager does not have weeks or months to evaluate applicant skills. Unless the hiring manager uses the hire-and-hope strategy, sales skills have to be evaluated in minutes or hours. If we have done our job right, we will know the mini steps that lead to maxi results.

Bottom line? If the analyst asks you to lump producers into groups and gives them all the same test, you are about to see your money pour out the door.

Does the Test, Test?

The only test that is worth anything is one that works for your job in your company, not one that worked for the company across the street, or a job with the same title, or matches a nationwide norm, or even a company in the same industry. It has to work for you.

Sometimes a validity study can be transported from one job to another, but that is only if you know for certain the two jobs are essentially the same. But if the market is different, the company environment is different, products and services are different, customers are different, or sales cycles are different, then how can any reasonable person claim XYZ scores predict cold calling, customer service, or sales expansion for your position based on one that is entirely unknown? Doesn’t that seem a little far-fetched to you?

The only time you can trust that another test will work best for your organization is to compare the job analysis from the other test to the job analysis for your job. If the two jobs are essentially the same, then use it; if not, you “pays your money and takes your chances.”

Give a generic personality test to salespeople and see what shakes out? Get ready to see a great big pile of belly-button lint.

Our Test Does Not Discriminate

In the U.S., at least, large organizations and Federal contractors are not supposed to reject qualified applicants based on age, gender, race, and so forth. This is called discrimination; but there is something else called adverse impact. What does adverse impact have to do with discrimination?

The legal definitions have subtle overlap, but for the purposes of this article, let’s assume discrimination generally means that an organization intentionally discriminates against certain kinds of job-qualified people?in hiring, promoting, training, and so forth. While adverse impact generally means the hiring system, even though it is job-related and professionally validated, unintentionally discriminates. In lay terms, think of discrimination as intentional and adverse impact as unintentional. For any better definition, see your local labor-law attorney to explain the details.

I consider discrimination unethical. Everyone deserves a chance to work in a job for which he or she is qualified. But here is where things get complicated. Government agencies examine discrimination at the group-level. Hiring managers don’t care much about group performance. They care about individual performance.

This raises a problem that all hiring professionals need to consider. By way of example, suppose 200 people apply for a job. One hundred are Lilliputians and 100 are Yahoos. At the group level, 70% of the Lilliputians are hired, while only 40% of the Yahoos make the grade. At the individual level, there are quite a few Lilliputians who are miserable workers, just as there are quite a few Yahoos who are top performers.

From the organization’s viewpoint, they only hired job-qualified people. From the government’s viewpoint the company discriminated against the Yahoos.

Who’s right? It’s hard to tell, so the government examines the organization’s:

• Professionally developed job analysis (to show hiring tools are based on job requirements and business necessity)
• Professionally conducted validation study (to show hiring tests and interviews accurately and consistently predict performance)
• Pass and fail results for Yahoos and Lilliputians at each step of the hiring process
• Proactive efforts to develop tests with less adverse impact on Yahoos

As long as the company has done its homework and followed generally accepted hiring practices as outlined in the “Uniform Guidelines” and “Standards,” it is not in trouble and will have hired all the best and most diverse applicants.

So what’s the problem? Some vendors claim their tests have no adverse impact. But research consistently shows hiring tests for jobs requiring problem-solving ability almost always does have an adverse impact when examined on a group level. Competent test vendors know this. Incompetent ones don’t.

Automated Resume Screens

What recruiter or hiring manager has not seen a brilliant resume developed by a blatantly unqualified candidate? And what about the marginal resume presented by a remarkable applicant? At best, a resume includes Kodak-moments recalled by the resume writer. At worst, a resume is an exercise in creative fiction.

Think about it. Every applicant is motivated to write just enough words to garner an interview. While every hiring manager wants to find someone who was an exceptional performer in the exact same job at another company doing the exact same work. Generality goals meet specificity objectives.

Sophisticated applicants know how to pepper the resume with keywords and qualifications that may be fact or fiction; different hiring manager’s screen resumes using totally different criteria for the same job; and, everyone makes massive inferences based on snippets of data. So. Tell me again. Other than keeping a few programmers in work, what is the benefit of automating resume searches?

Back to the Beginning

So here we are, back at the beginning. Tests are abundant. And if all you want to know is a score, anyone test will do. Good tests, however, ones that accurately predict job performance, are rare. You can trust a good test to produce good employees. You can tell the difference by following a few guidelines.

Avoid vendors that emphasize their non-discrimination aspects, “legality” or industry-wide applications. Assuming their claim is accurate (and I have yet to see one what was) users are responsible for their own test use. Vendors are off the hook.

Avoid vendors that want to give their test to two groups of producers and use the results to predict job performance. These represent bad science. Scientifically, this kind of study can only show whether the two groups are different, but it does not tell you why. And it does not tell you about individuals within the groups.

Avoid tests that are based on self-reports. Self-reported answers can be faked. They cannot be validated by outside sources. Self-reported tests are similar to resumes. They represent things the test-taker wants you to know about him or her. Making decisions about hard skills based on self-reported data requires a huge leap of faith that is generally wrong half the time.

Ask the vendor for a report showing he followed the ‘Guidelines’ and ‘Standards.’ This is your only assurance the test will be job related, based on business necessity and accurately predict job performance.

Web-based testing is in the same category as medicine was 100 years ago when heroin was good for you; there was no such thing as anesthesia; injections were unavailable; radioactive water cleared the mind; opium was a relaxation agent; blood-letting was commonplace; linseed, mustard, and soap were used as cure for infection; and sugar of lead was a common treatment for diabetes.

Let’s all work hard to move hiring into the 21st Century.

Every so often, I come across shameful hiring information included in newsletters. I always thought journalists were supposed to research their facts; however, in a recent career newsletter, there were three articles that immediately got my attention. If any readers come across articles like these, may I suggest you flame the author for reporting pure nonsense to both recruiter and applicant.

Secret Documents

One article suggested the “secret career document” approach. According to the author, this is a powerful technique created by one of California’s top marketing professionals. This professional allegedly guarantees applicants will immediately be in the top of the “must-hire” list for any position they seek. According to the author, handing this document to the interviewer literally encourages him or her to hire on the spot! What a concept! Where can I get one?

I don’t know about you, but I immediately have questions when I read this kind of stuff. For example, is the author clueless or shameless? Do journalists believe jobs are so simple that anyone can qualify? Or, are they recommending an elaborate program involving kidnapping and extortion?

I am not sure what to make of this claim. It seem like the writer believes interviewers are so inept they can be easily tricked by reading secret prose. He also seems to believe that anyone can have any job he or she desires just by writing a great letter. (The real truth is that he is probably taking a break from his regular job as a barrister finding beneficiaries for a $10 million Nigerian slush fund.)

Yes, smooth letters and job summaries can be helpful when seeking a job, but let’s get real. This stuff is silly. It puts interviewers in a bad light, makes every applicant think intro letters outweigh skills, and ranks right up there with promoting radioactive mouthwash as a cure for gum disease.

Freelance authors: Be careful with what you report. Recruiters and interviewers: Be careful with what you write because someone might just believe it.

Rehearsal Programs

Rehearsing the interview was another forgettable suggestion. These were recommended when encountering interviewers armed with insightful questions like, “What would your friends say your greatest weakness is?” or “What kind of tree are you most like?” or “What is the sound of one hand waving ‘get lost’?”

One consultant suggested foolish questions were often company policy. (Sure. And so is feeding everyone free coffee and locking the restroom doors.) If senior executives have enough free time to make silly questions “company policy,” then you can be assured they are wasting good money hiring poor employees. Zen-like questions might be good for achieving inner peace or staffing a Buddhist monastery, but they tell you more about the interviewer than the applicant. In fact, a wise Zen master might even say these interviewers have achieved a state of no-mind. I could buy that.

Don’t get caught in the leading-question trap or think you can discover hidden knowledge by asking questions that have unverifiable answers or are intended to provide deep insights into an applicants’ character. Smart applicants will outsmart you every time.

Beyond asking a few general questions to start the conversation flowing, make the majority of your questions job-related, specific, and hard to fake. If an interviewer cannot figure out what these questions are, then it is a sure sign he or she will default to the Sigmund Freud approach (but without the education, experience, or skill).

So, if “favorite tree,” “describe yourself,” and other unverifiable questions are at the top of your interview list, sign up for a course in behavioral interviewing. It will probably only cover about one third of what you need to know (the other two thirds are job analysis and standardized scoring). But, at least you will make a better impression on applicants than pretending to be a dead, 19th century psychoanalyst.

Bad Test Practices

Some people would like you to believe that motivation or personality tests are all you need to predict success. But, aside from unskilled jobs, doesn’t every job require some form of hard skills? Job performance is a two-sided coin: it takes both job skills and job motivation. Unmotivated, highly-skilled people are considered underachievers. People with different personalities perform the same. And people with the same personalities perform differently. Highly motivated but unskilled people are “train-wrecks.”

Does motivation accurately predict everything you need to know about job skills? Only if the sole product you have to sell is a personality test.

Academics make a serious distinction between traits, styles, characteristics, motivations, and so forth. From a hiring perspective, though, all we generally want to know is a person’s “AIMS.” That is, does the applicant have the “Attitudes, Interests, and Motivations” associated with job performance?

Applicant motivation is one of the harder areas to evaluate. It is easy to fake; unemployed applicants are generally motivated to get hired and tend to say nice things about themselves. So, as a general rule, questions about motivation tend to lead answers. Smart folks know how to fake well. In essence, you can never be certain whether applicants who answer motivational questions are being truthful, don’t have a clue about job requirements, or just faked you out. That’s a problem.

In my experience, motivation is accurately measured using hard-to-fake written tests that directly relate to job characteristics. “Directly related” means test items are not generic. The DISC, for example, is a four-quadrant generic test. This is OK for understanding that people are different, but it’s insufficient for predicting job success. Why? Jobs are complex. Dr. John Holland, for example, identified six separate job motivations: investigative, artistic, realistic, enterprising, social, and conventional. Holland observed that certain job characteristics appealed to different types of people (job skills notwithstanding). Additional research by other investigators eventually showed conscientiousness, extraversion, and not being neurotic affected job performance. In summary, a thorough evaluation of job fit and job attitudes requires measuring roughly nine dimensions…not four.

“Hard to fake” is another matter. It means the test delivers consistent results and test scores directly related to job performance. The DISC (and its clones), for example, is an “ipsative” test. Applicants are asked to select which adjective is most like them and least like them. At the end of the test, the “mosts” and “leasts” are summed and the totals manipulated to get a D, I, S, and C score. Nice and neat, right? Sorry.

Are all the adjectives equally weighted? That is, if Jamie selected 6 of the 24 dominant adjectives and Johnny selected a different set of six dominant adjectives, are their scores the same? Nope.

But that’s not all. Johnny and Jamie probably compared their six “D” adjectives to entirely different sets of “I” adjectives, “S” adjectives, and “C” adjectives. So, even the comparisons were different. Are Johnny and Jamie still equal?

Finally, after a little adding and subtracting, the test goes out on a limb and predicts that one profile represents a public face, one a private face, and one a back-up face, all from comparing four sets of 24 words. Please! If personality was that simple, psychologists would have stopped arguing about it years ago. Someone’s preference for 100 words predicts just that: a preference for 100 different words.

In summary, while Johnny and Jamie appear to have similar scores, they described themselves using totally different standards, used an insufficient number of factors, made individualistic comparisons, assumed equal weights, may or may not have matched job standards, and were backed by an extensive narrative based on limited data. Four factors do not cover everything you need to know. Comparative (e.g., ipsative) scoring doesn’t allow you to compare a person to a job. The DISC is probably OK for training, but it does not meet professional standards for hiring. Does this seem like a trustworthy hiring approach to you?

Conclusion

If you think finding applicants is hard work, then try testing their skills in a field characterized by an abundance of bad science, poor advice, and misinformation. It’s enough to make an applicant believe in secret documents and rehearse his or her interview answers before taking the interviewer’s DISC profile.

Get used to it: unless your organization hires everyone who applies, you are testing. Some people (even attorneys who should know better) vigorously deny that their organizations test applicants (pssst?interviews are tests!).

Whether an organization uses verbal questions or written questions, they both have the same objective: to separate qualified applicants from unqualified ones before spending big bucks on salary, benefits, and potential lawsuits. Tests are tests.

Now?let’s discover whether your test is working for you.

A Good Test

Separating a good test starts with reliability. Suppose an applicant takes a test on Monday and on his way out, you deliver a carefully aimed blow to the head sufficient to cause short-term memory loss (but not permanent damage).

After he gets out of the hospital, you invite the applicant back to take the same test a second time (with the promise of safe passage). Will he score roughly the same? That is, can you trust the scores to remain consistent from one time to the next?

This is called “test-retest reliability.”

Reliability means you can trust a test to deliver similar scores regardless of when it was taken. Otherwise, you would never know whether it was accurate.

Interviews, for example, are notoriously unreliable. Interviewers tend to like or dislike applicants; they may ask different questions of different candidates; they may think the objective of the interview is to get to know the applicant (wrong answer!); they tend to rate applicants based on personal appearance; and sometimes interviewers just talk about themselves. Interview test-retest reliability is pretty low.

Reliability is not limited to interviews. It also applies to many popular tests used in training, especially ones that measure personality type. Type-tests are fine for workshops and communication classes, but even some of the most popular ones are filled with reliability problems. Independent reliability studies show scores from a popular four-letter type-test tend to change from one time to the next. So, test authors, which score is the “real” score? The score on Monday? Tuesday? Last month?

Before you subject any applicant to a test, examine the vendor’s manual carefully and search for a section on “reliability.” You want proof the vendor knew enough to study the reliability of:

• Each test item (item analysis).
• All test items (inter-item reliability).
• The first test half compared to the last half (split-half reliability).
• The same people at two different times (test re-test reliability).

If you cannot find any reliability data, then your favorite test scores probably change from day to day. The next time you buy a pound of cheese, wouldn’t it be nice to know you were really getting the weight you paid for?

Hopefully you see that unreliable tests are a dead end, especially since most organizations want their tests to predict performance.

Using Test Scores for Prediction

Predicting job performance means that a reliable test score is directly related to job performance. The word “directly” means two things. First, it measures something that affects job performance. Second, the scores correlate with ratings. A typing test, for example, is clearly linked to jobs that require keyboard skills. If your organization still has a typing pool, the scores probably indicate the amount of work a typist can do.

But are keyboard skills always linked to job performance for management? Should we fail candidates who could learn keyboard skills in a few weeks or months? Do we know if applicants are physically unable to operate a keyboard?

Accurate prediction is called “validation”?and if you thought reliability was complicated, you ain’t seen nothin’ yet! Validation requires knowing clearly what skills are necessary for the job, and doing sufficient analysis to show test scores are statistically correlated with job performance (i.e., the test content and job requirements are causally related).

Otherwise, you are predestined to turn away qualified people and hire unqualified ones. Is that wrong-headed or what?

Why Should I Care?

If your objective is finding and filling, then you probably don’t. Stop right here, get some coffee, and don’t send me any nasty-grams. I assure you reading this article will be a colossal waste of your time.

However, if cutting turnover in half, doubling individual productivity, reducing training expenses, and building a solid base on future-qualified employees is attractive, then you need to know this. These claims are all normal for an organization that uses reliable and valid tests. Why? Their tests screen-out unqualified applicants. In case you are wondering, only about one applicant in six (on average) can pass a series of validated tests. Put another way, only about one applicant in six can demonstrate skills required for the job.

Ever hear about the 80/20 rule?the one where 20% of the people produce 80% of the results? It’s amazingly close to a one-in-six hiring ratio. Think about it. So if you care about making the biggest splash ever in the company pool, then continue reading.

A Bad Test

A bad test is one that an organization uses consistently, is backed by folklore and plenty of personal anecdotes, but has never been critically evaluated. Bad tests usually come out of corporate training programs. That is, a workshop participant who answered 10 questions about being a thorough planner was “amazed” when the test reported he or she was exceptionally organized. Next step?.use it for hiring!

Folks, personal agreement with test scores is not a reliable and validated way of predicting job performance. It is only a summary of how someone describes himself or herself. It is a self-reported description. Is the person actually as organized as he/she says? Or are they faking? If they are not faking, is organization important to job performance?

Defining the Job

This is a tricky area. The secret is to define the critical skills that directly affect job performance. This might include learning ability, problem-solving skills, persuasiveness, and so forth. The key to defining job requirements is to identify behaviors leading to job success or failure. It sounds weird, but you don’t look for results, just the behaviors that lead to the results.

If you cannot clearly define the key job skills, then there is nothing to test. The 1978 Uniform Guidelines suggest job competencies be based on job requirements and business necessity. I don’t know about you, but that sounds pretty good to me. Amazing! The government recommends organizations test for job requirements and business necessary. If anyone out there can suggest something better than basing a test on job requirements and business necessity, I’d like to hear it.

To reiterate, your test first has to be reliable. Then you must know what to explicitly measure. To make sure the test works, determine whether test scores predict job performance. We call this step “validation.”

Throw It On the Wall and See What Sticks Approach

Here is a sure clue to wrong-headed hiring practices. It goes like this. A vendor has a general personality-style test (we’ll make a fanciful assumption that it passes professional reliability standards). The vendor herds high producers into one group and gives them the test. He examines the averages and declares, “Yea, verily, these scores doth become our target!” (vendors like to use old English?it sounds so classy!).

Whoa?not so fast.

How does one define high-producer? By results or by actions that lead to results? It makes a big difference. Individuals in the high-producer group could have used different skills to get there. Some might be good politicians. Some might be very smart. Some might be taking credit for others’ work.

What about the confusion between correlation and causation? Just because ice-cream sales and shark attacks are correlated does not mean that one causes the other. Almost anything can be correlated, but not everything is causal. If you sort through enough garbage, you are likely to find correlations between cookie wrappers and hotdogs. So what? Your goal is to find a correlation between hotdogs and hotdog buns.

The “see what sticks” approach has a few natty problems. Sure, it looks scientific, but what good are decisions based on wrong-headed performance criteria, wrong-headed clustering techniques, and wrong-headed statistical analysis?

Job-Match Approach

The job-match approach is scientifically similar to the “see what sticks” approach, except worse. Some types of tests say certain occupations have similar styles: Introvert Sensing Thinking Judging (ISTJ) for example.

Before you use this stereotype for hiring, ask yourself if all the people in the same occupation do the same thing, or do they all perform equally well? Did their personality style cause them to be an engineer? Are these folks extreme ISTJs or are they marginal ISTJs? Do their organizations all have the same objectives for the job?

Conclusion

Everything starts with the human elements of job requirements and business necessity. Human elements are seldom included in job descriptions or job evaluations. You have to dig for them. If you cannot test/interview for specific human elements, your tests will probably be inaccurate.

All selection tests have to pass rigid standards for reliability and validity. Reliability means the test delivers consistent results time, after time, after time. Validity means the test scores accurately predict job performance and should be done carefully.

It is a grave mistake to assume any group of performers has equal skills. For example, some salespeople are great repeat sellers, some are great cold callers, and others are great service people.

They all might be high performers but for entirely different reasons. It is a big mistake to assume characteristics or traits correlated with performance actually cause performance.

It’s a common assumption. A good sales manager should first be a good salesperson, right? Wrong. It is a big jump from being a skilled “doer” to being a skilled “coach of doers.”

In many cases, the top sales person is an enigma. Salespeople are ego-driven and competitive and want to be recognized and rewarded. But like a cup with a hole in the bottom, no matter how much water you add Monday, it needs to be refilled by Tuesday. Top salespeople are often driven by ego-validation that comes from each sale and public recognition, and it never stops.

On the other hand, a top sales manager enjoys stepping back and letting others take credit. He or she takes pleasure in the success of subordinates and has a “smaller cup” to keep filled. Sales managers are like gardeners, tending and nurturing promising salespeople, having the patience to wait and watch them mature.

Defining Top

“Top” and “skilled” do not necessarily mean the same thing. Top salespeople are often measured by dollars or accomplishments; that is, the score at the end of the game. And like most folks, bosses tend to ignore how he or she got to the top.

For example, I once objected when a salesperson told a client we could do a job for 50K. I knew from experience the quote was about 1/5 what the final project would cost. The salesperson griped to the president; I was told to do the work; the salesperson won a sales award; and the client went wild when he saw the final bill. (Yes. It was my fault. I was unable to control the 80% gap. Shame on me. Mea culpa.)

Not only do top-dollar salespeople usually bend the truth to fit their ego needs, they often operate on automatic pilot. That is, top producers seem to instinctively know what to do and what to say. It is one of their gifts.

But instinct is often not something that can be broken down into coaching activities. The clue to an instinctive salesperson is, when someone asks for advice, they reply, “Watch?and learn.”

If learning were that easy, then we would all be experts just by watching a videotape. This instinctive condition is called “unconscious competence.” In other words, the person is unaware of why he or she is good; it just happens naturally.

A skilled producer, on the other hand, knows exactly what to do and when to do it. He or she can break down sales into small clusters of behavior such as initial relationship skills, questioning and discovery techniques, strategic analysis of the situation, and so forth. The big difference is the skilled performer is conscious of his or her competence: he or she knows when to choose certain skills. It’s not an automatic reaction; it’s a conscious decision.

The combination of sales-ethical behavior and general awareness of the situation generally makes the skilled salesperson successful, but not a top-dollar producer. Truly skilled salespeople put full effort on (this is a very important point) solving customers’ problems, not persuading people to buy. As such, the skilled salesperson will walk away from bad business because it is bad business.

Technical Knowledge

Both the skilled salesperson and skilled manager are knowledgeable. They know their product or service thoroughly; they know the marketplace; and they understand the world where their customers’ live.

A top-dollar salesperson, on the other hand, often blows off product knowledge as unimportant. These blowhards often claim they can “sell anything to anyone” (based entirely on their own opinion of themselves). They don’t seem to understand that people generally dislike being sold something they don’t want or need; further, people who feel manipulated make poor repeat customers.

Sales Skills

The skilled manager thoroughly understands how the sales process is divided into skills-sets that float back and forth. Relationship skills are necessary to open the dialogue and keep communications flowing. The main relationship skill is the art of helping the prospect or client feel comfortable.

Primarily, it includes words and actions that create an environment where the prospect feels he or she is the most important person in the universe. It includes projecting confidence, product and market knowledge, true empathy, and personal understanding.

Relating skills are brought into play when meeting, asking questions, encountering resistance, and giving assurances. If you think genuine relationship skills are difficult for an ego-centered person, you are right.

Fact-finding and discovery skills are the second most important skill-set. These include all the open and closed-ended questions associated with learning about the prospect’s environment (i.e., what is happening in the marketplace, how it affects him or her, what’s happening in the organization, their personal fears and aspirations).

The objective of discovery skills is for both prospect and salesperson to come to the same conclusion at the same time: “AHA! Here is a problem worth solving!”

A fly on the wall witnessing these two skill sets would see people chatting and discussing work. There would be no pitching. The salesperson would be gently fact-finding, offering helpful suggestions, and encouraging the flow of conversation. His or her objective would be to discover whether the prospect had a problem that needed a solution.

For example, it won’t do for a bypass-surgery salesperson to approach a weight-challenged prospect and ask, “Have you ever noticed your head is much too small for your body?” Sensitive people might find this question offensive.

A skilled salesperson, on the other hand, would take time to know the prospect as a human being and through discussion, both would agree that weighing 500 pounds was not only unhealthy and uncomfortable, but he or she would never be able to compete in the annual cardiac ward break-dance competition. Now we move into the third skill-set: sales nirvana!

Sales nirvana is when the salesperson finally gets to take center stage. He or she gets to talk. This is the time to recommend solutions, discuss why they will solve the problem, why they are the best of all solutions, and why they will continue to be effective both now and in the future. Eighty percent of all salespeople I evaluate tend to think this is the only part to selling.

However, unless our weight-challenged prospect decided to drop 65% of their body weight at 1 p.m., and a bypass-surgery salesperson walked in with a solution at 1:05, being “at the right time and the right place” takes work.

Of course, not all sales calls go this smoothly, and most prospects waffle back and forth between relating discovering and solving until they reach a decision. Skilled sales managers recognize these separate stages for what they are and know exactly what to do next. That’s what makes them good coaches. They are conscious of the sales process and competent to act (or coach) accordingly.

On the other hand, top-dollar sales producers dance through the sales process like Savion Glover, Gregory Hines, and Fred Astaire all rolled into one. “Watch and learn,” they shout as their blazing feet disappear into a blur of motion. Right.

Conclusion

There are a few very important points to make about salespeople and sales managers. For one thing, in their efforts to keep top-dollar salespeople happy, bosses often unwittingly destroy them by giving them a promotion. (If the bosses are lucky, the incompetent sales manager will quit before the rest of the sales force is decimated). Wouldn’t it be better to let salespeople realize beforehand that management is not a promotion, it is an assignment to a different job requiring entirely different skills?

Another important point is sales skills and management skills are seldom interchangeable. When hiring a salesperson, look for someone who thoroughly understands each step in the sales process, not someone who is skilled at selling wastebaskets (or ashtrays) to the VP of sales. Don’t fall in love with the sales pitch; selling is more about fact-finding than persuading.

When hiring a sales manager, look for someone who not only knows the sales process, but is able to develop specific skill-sets among the existing sales staff. Forget about turning top-producing salespeople into skilled managers; this seldom happens.

After all, your head is not too small for your body, is it?

Decreasing turnover and increasing productivity can be a natty problem. Solutions usually come in one strength: weak. That is, incentive programs, public awards, and social get-togethers generally fail to make a long-term financial impression.

I won’t claim that employee appreciation programs aren’t a nice touch; after all, everyone wants to feel appreciated. But such recognition does not solve the real problem.

Care and Maintenance of Cube Farms

Jobs are considerably more complicated than they appear. The recruiting and hiring process is easy to understand. The hard part begins after hiring. That’s when employees endure a manager’s personality and level of competency; receive varied compensation and benefits for doing similar jobs (generosity and equality are not a widely shared corporate objective); encounter daily job frustrations (lack of resources, conflicts with other people and departments, conflicting policies); and worry about surviving the next reorganization.

That is what we forget about organizations. Companies, government agencies, and non-profits are not buildings or equipment. Organizations are people. That’s why we call them “organizations.”

Organizational research confirms that excessive turnover is most often a function of the wrong people in the wrong job. Specifically, managers who do a poor job choosing the right seed and forget to nurture their Cubies.

The Deciders

Excessive turnover and low productivity are symptoms of management treatment and decisions.

We’ll start with executives and middle managers?the folks who make the big decisions. Although these candidates need it the most, they avoid assessment like a televangelist avoids a sunshine audit.

Executive decisions require considerably more than the average level of skill. They need an almost superhuman mental ability to anticipate opportunities and avoid unexpected problems, combined with a personal charisma to make it happen.

In B-school terms, this is called “expert knowledge” and “leadership.” B-school case books are filled with hundreds of examples of company executives who lacked expert knowledge and zigged when they should have zagged. Just check out the business section of your local newspaper for the massive mistake of the month.

Once you get past an executive’s glowing resume, dig for details. Most important, try to understand the skills and motivations he or she will bring to the job. Often these will not be evident in the resume, nor will they be evident in the interview. Both usually address results, but “results” are often not the same as skills. Think of results as the score at the end of the game and skills as how the game was played. You need to know skills.

Key skills that should be assessed before turning over the family jewels to an executive include his or her ability to make reasoned decisions when information is totally lacking; his or her ability to persuade (not cajole) subordinates to take action; and most important, his or her hidden attitudes, interests, and motivations.

When people get promoted (or elected) to positions of power and authority, the first casualty is usually humility. Dr. Bob Hogan has written extensively on how a large number of executive failures can be attributed to narcissism that blooms after a promotion. That is, the executive or manager thinks he or she is above reproach. Just think of Leona Helmsley, Ken Lay, Bernie Ebbers, and (more than few) high-profile politicians who thought they deserved special treatment.

Pat, Howie, and Roger

Assuming your executives are not the poster kids for bad behavior, and assuming working conditions in your organization are better than the average troll mine, what other factors do you think have a substantial effect on turnover? Did you guess incompetent front-line managers?

Employee surveys generally say front-line managers are the greatest source of dissatisfaction about their job. I don’t know about you, but I have worked for some real sociopaths. Some were paranoid about losing their job; some felt subordinates should be human-shields whose purpose was to make the manager look good; and some were totally clueless about what their subordinates did.

Some of my ex-managers’ priceless comments still ring in my ears. For example, after 15 months of no departmental meetings, Wrong-Way Roger exploded with, “Meetings? You know what to do! If I held meetings, I would be doing your job!”

Another manager, Oblivious Pat, claimed he could learn his PhD’s job within a few months. And Horrible Howie insisted his PC be programmed so all he had to do was press one button to get anything he wanted.

While some might claim these people were dipsticks in the motor of life (I would never say that) others would see them as shining examples of a management selection system that put the wrong people in the wrong job. Wrong-headed Roger should have been kept apart from human beings. Oblivious Pat should have been an auditor. And Horrible Howie should have stayed in school.

Management, especially front-line management, can have more effect on turnover than any other system. If your organization suffers from high turnover, the first place to look is front-line management.

Key Management Skills

Front-line management jobs come in all shapes and sizes. On one hand, there are people with a management title, but their job involves “doing,” not coaching. Their title could be a reward for good performance, an ego boost, or strategic positioning. No matter. This kind of manager does not manage. He or she is an individual contributor and should be evaluated for contributor skills, not management skills.

True front-line managers are usually coaches. That is, they have the responsibility for managing, developing, or correcting the activities of the people they supervise. Front-line managers get things done through people, not by doing the job themselves. None of the managers in my examples were able to manage, and their interactions with intelligent human beings were caustic. Their behavior drove turnover.

Front-line managers do not have to be expert in the jobs they supervise. They just have to know it well enough to give direction and guidance. This is not an easy skill to acquire; while some argue it can be learned, my experience evaluating managers of all types show few people have the skills to coach effectively. In practice, they tend to dance around the issue or make veiled threats. None of which helps their direct reports get better.

Manager/Coaches = 1 in 7

When I evaluate managers, only about one out of every seven can intelligently question, discover problems, make appropriate suggestions, and get subordinate commitment. Six out of seven flounder?mostly in the coaching arena.

Use better standards to select and promote new ones by evaluating candidates’ abilities to do the following:

• Plan and organize complex activity.
• Coach and develop others.
• Be able to think analytically and see both the forest and the trees.
• Be able to perform the job, but not necessarily be an expert.
• Have the attitudes, interest, and motivation of a manager.

Always remember first-line managers are cited as employees’ greatest source of stress. And stress can be a significant reason for turnover. Incoming and first-line management promotion decisions are the easiest to tackle. Simply forget about “promotions as a reward” and focus on “promotions based on job skills.”

Evaluate prospective front-line manager job skills by interviewing about past behaviors, administering tests and exercises, and watching the person perform in coaching simulations.

If you want to start reducing turnover, make sure every incoming executive and manager has the right skills for the job.

I really don’t like being a resident doomsayer, but organizational hiring and promotional practices are generally so abysmal that I am compelled to make it a big issue. Take EEOC tracking, for example. Most people think all they have to do is send in routine normal reports. Not so. They should be monitoring adverse impact throughout their entire hiring process.

According to the EEOC, and best practices in general, each employer should be monitoring adverse impact by using the following rule of thumb:

1. Calculate the pass rate of your majority demographic group.
2. Calculate the pass rate of your minority demographic group.
3. You have adverse impact if the fail rate of the minority group is more than 80% of the majority group. For example (same job): male pass rate = 50%, female pass rate = 35%. Test: ….80% x 50% = 40%; however, only 35% of females are hired; therefore, the organization is guilty of adverse impact against females.

Does that sound complicated? Well, maybe this article will help clear it up.

What’s an “Applicant?”

This used to be quite clear: applicants were people who showed up and filled out a form. Now, no one is quite sure when a person becomes an applicant. Is it when 100 people respond to an Internet post? When 50 people are phone-screened? When five people are invited for an interview?

Although it will probably be years before there is a universally accepted definition, every organization should have a formal policy that makes a reasonable argument for when someone qualifies as an applicant. Your local labor lawyer should be able to help you draft something that works. Just be sure to follow policy until the legislators tell you otherwise.

Every Organization?

Generally, only larger organizations (e.g., with 15 or more people) need to be concerned; however, as your public presence grows, so does the public’s perception of your financial pockets. Remember, just because most challenges are settled, not court ordered, you don’t get off cheap. Also, some organizations are exempt, so be sure to check with your local labor law attorney.

What’s a Minority?

Certain groups are usually people who are protected by law: the disabled, different races, females, 40 and over, people with different beliefs, and so forth. Check with your local labor lawyer.

Why a Labor Lawyer?

You wouldn’t go to a podiatrist if you had a heart problem, would you? Law is a pretty big field and corporate attorneys are generally skilled in contract and business law. Labor law is a specialty. You usually get better advice from a specialist than a generalist.

Now, one more problem. My experience with corporate and labor lawyers is they know the law, but they don’t know how to build HR systems that keep you out of trouble. In fact, in the last 17 years of practice, I have only met one attorney who understood how to build a legally credible hiring system. The rest wait until the end of the meetings and quietly ask me for copies of the DOL Guidelines.

What’s Adverse Impact?

Adverse impact is when a selection system (applications, interviews, tests, resume screens, sources, and so forth) rejects too many legally protected applicants using the 80% (i.e., the 4/5) rule. But that raises another question: is adverse impact illegal? No.

Organizations have the right to reject unqualified applicants (even people who belong to protected groups) as long as they can produce the following:

1. A professional job analysis.
2. Proof that scores on tests predict future job performance.
3. Evidence that tests are based on job requirements and business necessity.
4. Evidence they constantly look for better ways to reduce adverse impact.

Remember, interviews are tests.

Wrong-Headed Ideas

Some folks argue that organizations should “dummy-down” their jobs so anyone and everyone will qualify for a job, regardless of protected group membership. To the best of my knowledge, most of these folks are either politicians running for public office, politicians trying to keep their job, or people who have never run a business.

Unless you are willing to hire everyone who applies and promote everyone who asks, you probably want to use some kind of skills test.

Statistical Analysis

Adverse impact is determined by clustering people into groups and looking at statistics. This is not always the way the world works. For example, it would assume that every female, older candidate, member of a minority group, etc., are equally skilled for a given job. Common sense tells us this is seldom the case.

When it comes to statistical analysis, it takes big numbers to get trustworthy data (about 100 people in each tested category). Testing for adverse impact means you need at least 100 people from each racial group; 100 people from each age group; 100 people from each, well, you get the picture. These kinds of numbers often take a long time to accumulate, and in some hiring locations, may take a generation or two!

Oh, and if you want to get technical, the last time I heard, the EEOC examines your hiring data using cross-tabulation and the Cochran-Mantel-Haenszel statistics.

And you thought the 80% rule was complicated!

What’s Next?

Suppose you fail the adverse impact test. What happens then? You break apart the hiring process and examine the pass rates at each hiring-decision step (i.e., resume review, phone screen, test, interview). Don’t collect this data today? That could become a nasty problem in the future. Are third-parties exempt? Sorry, third-party recruiters, as well as anyone else associated with screening and hiring, are affected by these rules.

What’s the Risk?

While the DOL does not force organizations to follow the 1978 Uniform Guidelines on Employee Selection Procedures, it uses the guidelines as standards for adverse impact charges.

Few court cases, you say? Check out this link to the EEOC website and make your own decision. Be sure to check out the settlements page. The real cost is bad hiring decisions. Researchers estimate bad hires can cost from 20% to 50% of base salary, each year! (By the way, employee selection is not limited to hiring. It also includes promotions, terminations, and a host of other selection decisions).

Blending Best Practice with the Guidelines

I’ve presented the EEOC as a big stick throughout this article. At its foundation, the guidelines are not only a government recommendation, they are a best practice. That is, every organization can reap financial benefits by following them.

Who could argue against every job having a trustworthy set of basic, measurable competency standards? Or if every test and interview were as accurate and relevant as possible? Of if every job-qualified person was treated fairly, and every hiring and promotion system was monitored? Finally, who could argue against tighter hiring standards that lead to better employees?

What it Takes

Hiring is not about sourcing. Sourcing is only one part. Hiring is about defining standards, accurate measurement, and monitoring. In other words, it is about getting people with the right skills in the right job.

HR is generally a transaction-oriented department that processes work orders, mass- screens candidates, conducts workshops, and processes benefits and paperwork. These commodity functions can be easily outsourced. Imagine the value HR could bring to management if HR truly controlled the quality of employees who were hired and promoted?

Once a week I get called by someone, usually an administrative assistant, asking if I have a personality test. When I try to get more information, she usually tells me her boss wants the test and she just wants the price.

I tell her, “Sorry. I sell solutions, not tests.”

She hangs up absolutely convinced I am a jerk.

Putting aside the fact she’s probably right, let’s examine her request.

When bosses want to buy a hiring test, it’s usually because they want to reduce hiring mistakes. With typical don’t-know-what-they-don’t-know aplomb, they send their hired help on a fishing expedition to find a test to “fix” what they don’t know. Then things get problematic.

Why is it so important to make a distinction between selling tests and selling solutions? Because selling tests benefits the seller; selling solutions benefits the buyer.

Hiding in Plain Sight

Testing falls along a continuum:

• Type 1 (chance): Hire anyone who applies and fire people who fail the job-test (a popular variant is to aggressively recruit based on personal opinion and cull the employee base 10% annually).
• Type 2 (also, chance): Ask questions, give tests, compare applicants to each other, and disqualify applicants based on choosing the wrong animal, tree, or legume they would most like to be.
• Type 3 (about 25% accurate): Study the job thoroughly, ask hard-to-fake-questions, and systematically compare results to job requirements.
• Type 4 (about 50% accurate): Study the job thoroughly, give a solid, legitimate, and validated test that directly applies to job performance and systematically score results.
• Type 5 (about 75% to 80% accurate): Combine several validated methods that cover the entire job domain.

Buy-a-test people are like patients who walk into a doctor’s office expecting to be treated but not saying where, why, or how it hurts. It’s hard to recommend a solution when there’s no clear understanding of what needs fixing.

Hiring and Performance Theory

Best practices start with having a sound theory of performance behind the test. This is psycho-babble for: it’s always a good idea to test for something that makes a difference between high and low performance. Basically, this means if you use a test like the DISC or MBTI to make better hiring decisions, you better have hard data to support your decision (especially since reputable vendors of both tests recommend against using the DISC or MBTI for hiring).

Putting people in little style boxes may be fun stuff, but it takes homework to prove XYZ style actually predicts job performance (it usually doesn’t).

Are theories behind the DISC, MBTI, and MMPI based on job performance? No. Read their manuals. These tests describe personal differences, not job skills. Don’t make hiring decisions based on any test score unless you could produce data proving it predicts performance. And if the test authors advise against using their test as a hiring tool, then it’s probably a good idea to follow their recommendations.

What if the authors suggest only using the test to help make an informed hiring decision? Huh? What part of “testing” is unclear? If a score is used to “help” make a hiring decision, then they are being used as a test.

Next Step

Ok, suppose the test is backed with a sound job performance theory?what’s next? Face validity, reliability, and internal construction.

Face validity means the test questions and format should resemble a legitimate test. That is, the test should never include invasive questions about the subject’s personal lifestyle choices (e.g., do you belong to a spouse-swap club?) nor questions that have little to do with job performance (such as asking the person to stand on their head and sing The Star-Spangled Banner in a public place). Not only is a lack of face validity bad test practice, it makes the organization look silly and unprofessional.

Reliability is more complex. A good test should produce similar results time after time, like a bathroom scale. If you weigh yourself on five consecutive days, you would expect your scale to produce similar results every day, not vary between 100 and 200 from one day to the next. Likewise, a hiring test should produce roughly the same results from one time to the next.

Reliability also applies to interviews. If three interviewers asked the same questions of an applicant and they heard dissimilar answers, then the interview can be considered unreliable. There are many popular tests that produce unreliable results. How do we know this? University researchers are paid to run investigations.

One sure sign of unreliability is short tests that produce long narrative reports, or long tests that try to measure too much, or tests that rely on opinions, or complicated scales.

Does the Test Really Work?

If you’re certain the test is based on a solid theory of job performance and it’s reliable, then you are ready to determine whether it actually works. This means gathering numerical evidence that good scores actually predict satisfactory job performance and bad scores don’t.

This is no time for personal stories and anecdotes. Vendors and users alike have personal stories about what works and what does not; however, validity requires hard facts and abundant evidence the test works for you, in your job, in your organization. After all, that’s what you really want to know, isn’t it?

Validity is where the rubber meets the road. It is the point of contact between applicant skills and job skills. All the theory and reliability in the world is a waste of time if the test scores do not lead to better hires and promotions. Validity is the “solution” part. No validity, no solution.

Back to the Beginning

In the beginning of this article, I mentioned the elusive search for a test to make better hiring decisions. Here’s the bottom line: Know first what you want to fix. It could be a turnover, training, performance, or a hiring problem. Within each of these areas, the problem could be traced to intelligence, planning, interpersonal, or motivational skills.

Each of these areas has different solutions. In fact, it could even be a bad manager, bad management, or bad environment problem. There is no way to fix a problem if you don’t know what’s broke.

Once you know what to fix, find a trustworthy, consistent way to measure it. This could be a job preview, one-on-one simulation, structured interview question, smart application blank (i.e., an application blank with specific questions developed based on a statistical study of turnover or productivity), biographical data, tests, case study, or demoting incompetent managers. Each method comes with a complete set of pros and cons. Nothing is perfect, but there are some methods that are considerably more accurate than others.

The last step is to make sure the measurement tool works as intended. All of the above work will be wasted unless you know for certain your test(s) deliver trustworthy, consistent, and accurate results.

Pick one: a test or a solution?

Sure, it’s easy to say engineering, legal, IT, or actuarial jobs require technical degrees. People in these professions need a substantial amount of education to practice their trade. But we all know from watching folks in these professions that it takes more than a sheepskin to be successful. Sometimes, it takes certain personality factors to make a good job fit.

Job performance is a two-sided coin. On one side, all the hard skills required for the job, and on the other, all the social factors. For the purpose of this article, we’ll assume personality is on the social factor side.

The important things to ask are, “How do we decide which personality factors are important?” and “How do we measure them?”

Wrong Way

A pre-screen test (i.e., interviews, tests, resume review, application form) is supposed to determine whether the applicant is qualified. In a perfect world, good scores predict good performance, and bad scores predict bad performance (assuming we hired applicants with bad scores).

Well, there’s a right way and a wrong way to link scores to future job performance. Unfortunately, the wrong way is the norm.

The wrong way comes in two varieties:

1. Someone likes XYZ test, gives it to every applicant, and plays amateur shrink.
2. Someone gives a one-size-fits-all test to current employees and correlates scores with job performance.

Both varieties are filled with mistakes. I call these WW1 and WW2.

WW1 is common among most organizations I know. WW2 is common among folks who might have taken a class in statistics, but skipped the class in measuring human performance. Either way, both WW1 and WW2 turns away good employees and hires bad employees. Of course, you don’t have to take my word for it. Just do a controlled study.

What is This Thing Called Performance?

If we want to use a pre-hire test, common sense says to first determine what factor we want to measure. Second, choose an accurate and trustworthy test that measures this factor. Third, make sure the test predicts job performance. Sound simple? Read on.

Some people think they can measure “performance” by looking at performance appraisals or supervisor ratings. However, we all know most performance ratings are primarily personal opinion shaped by friendships, power, ambition, or sucking- up (sucking-up is a technical term for getting other people to think you are actually competent at what you do).

As a result, most performance review data means:

1. The employee was unskilled at shifting blame for his or her mistakes and now must wait for the rater to get Alzheimer’s.
2. The employee is highly skilled at shifting blame and charismatic?and thus, is well-liked by the boss.
3. Employees who are skilled at corporate camouflage.

In general, any data that can only be evaluated at the end of a long performance period is error-prone. Sales volume, for example, is fuzzy because it’s a function of persistence, fact-finding, learning, strategizing, presentations, adapting to buyer personalities, economic conditions, market conditions, and so forth.

To use an analogy, if you want to measure the quality of grapes in a fruit salad, you cannot put all the fruits into a blender, press mix, and come back six months later with your grape-o-meter.

Performance data should be easy to see and easy to measure.

Big Nets Often Contain Big Holes

There are some folks who think they can give a multi-factor personality test to employees, and then examine the results to identify correlations. Sorry. No cigar for them, either.

Let’s assume for a moment these folks got the performance thing right. Giving one great-big test to everyone and looking for results is a major mistake, primarily because a correlation is not always a causation. In other words, just because two things are statistically associated does not mean one causes the other.

Consider the famous correlation of rising skirt hemlines with rises in the stock market. If one actually caused the other, today’s hemlines would probably be located somewhere above the waistline, causing most male traders to become so distracted a market crash would be inevitable.

Statistics is deaf, dumb, and blind. It can show how two factors move with relationship to each other. Only a human being can decide whether one factor causes the other. For example, global organization or teamwork factors may correlate with performance, but if organization and teamwork don’t cause performance, then hiring people based on these scores will reject qualified candidates and hire unqualified ones.

How Many Factors?

The DISC is a widely used tool. Just check off a few dozen adjectives and you can get a mini-novel. It looks good, but is the DISC really as comprehensive or accurate as it looks? You decide.

The DISC is based on a two-factor theory that is almost 80 years old. It states that personality is a function of being either active or passive in a friendly or unfriendly environment. Does this sound sufficiently comprehensive to define your job?

I suppose hiring everyone who described themselves as mini-Napoleons would produce an employee base of toy soldiers who continually fought for power. However, would that be productive?

I once visited a company where the HR guru only hired people who excelled at teamwork. After a few years, it had 300 employees who would not schedule a meeting unless everyone could attend, make a decision unless everyone agreed, and not confront a production problem because it might hurt someone’s feelings.

Did they get the results they expected? No. They got the results they measured.

University research shows it takes about nine to 10 factors to predict job performance. Six or seven are based on job fit and three are based on job attitude. It is probably a good idea to check your test vendor to see if the test you are using was developed based on this research. If not, it probably won’t either predict the performance or attitudes you need.

Statistics

As I said before, statistics is deaf, dumb, and blind. It measures associations, but only under the right conditions. For example, data sets smaller than 25 people are filled with too much individual information and not enough group information.

That is, comparing the characteristics of 15 high producers with 15 low producers is probably going to contain a substantial amount of error.

The bottom line? Trustworthy statistics needs large numbers of people to be accurate.

What about the size of correlation? Should we shout and celebrate a correlation of .30? Does it mean we have a 30% relationship? No. It means we have a 9% relationship. The technical details would cause John Wayne to weep, but correlations have to be squared to learn how much performance they predict. People who do not understand statistics are easy to fool.

Wrap Up

Personality can be a very important thing to measure pre-hire because it can provide considerable insight into job performance. However, before you can trust a personality test to predict performance in your job, you have to clearly define what you want to predict; identify the personality factor that causes it; find a test developed specifically to measure that factor; either conduct your own study or transport someone else’s work; and finally, realize that personality is only one part of the puzzle.

Otherwise, if your organization is still using WW1 or WW2, it is probably turning away good employees and hiring bad ones.

A large number of readers recently rushed in to confirm that military hires were “slam-dunk” employees. Almost every story was backed with a personal anecdote and criticism was branded downright unpatriotic.

I spent a few years in the military and have the highest respect for those who put their lives on the line. I encourage every employer, if given a choice between two equally qualified candidates, to hire the vet.

However, assuming a person is job-qualified because he or she was in the military can seriously affect recruiting decisions. This is a human judgment error.

Halos

Human beings make sweeping conclusions every day based on virtually no data. In lay terms, this is called either the “halo” (when an entire assumption is based on one positive trait) or the “horns” effect (when an entire assumption is based on one negative trait). Advertising companies throughout the world are masters at using halos and horns.

Halos occur every time a company puts a celebrity in front of a camera. For example, Catherine Zeta-Jones tells us her cell phones are great while Sam Waterstone recommends investment advisors. I have no idea if Catherine Zeta-Jones even knows how to open a flip-phone or if Sam Waterstone can do basic math. Nevertheless, because Catherine is beautiful and Sam is incredibly credible, halos encourage us to assume they are also phone and financial experts.

Recruiters and hiring managers are influenced by halos as soon as they form a subjective opinion about an applicant. Think about the first-impression rule. The interviewer assumes that because the applicant looks good, sounds good, or is tall, he or she is job-skilled (cue Ethel Merman’s rendition of “Everything is Coming Up Roses”). The halo force is so strong that we forget there’s no substantial evidence to support our opinion. But that doesn’t make any difference. Right or wrong, never in doubt!

Horns

“Horns” is over-focusing on the negative. It happens when we see one typo on a resume and conclude the candidate must be illiterate.

Horns are popular tools among political spin doctors as well, but the important thing to know about horns is a little bit of negative information can be just as misleading as a little bit of positive.

On the Job

If you’re a recruiter who only sources candidates and is totally unconcerned with skill qualifications, then the effect of halos and horns on your decisions is unimportant.

However, if you make recommendations about whether a candidate has job skills, and expect to be paid for your professional expertise, then you cannot afford to make sweeping generalizations based on halos and horns. Doing so means unqualified people will be hired and qualified applicants will be turned away.

Let’s examine “halo” in a military context.

To listen to some people, military people are characterized by discipline, character, and leadership. These are all admirable qualities. But the military job I had was also characterized by rigorous physical demands; high turnover; critical and often life-threatening tasks; a massive influx of inexperienced people; following orders; sacrificing anything to complete the mission; a set of standard operating procedures for everything; and severe penalties for disobedience.

It’s a good idea to remember the predictive accuracy of past performance is highest when it resembles future performance. That is, the greater the difference, the more opportunity to make a mistake. Past experience in the military is a highly accurate predictor for future performance?in the military. If an honorable discharge was all one needed for job success, every veteran would be a captain of industry.

Does military experience ensure job success? Nope. Job skills do.

Be careful. Past experience is a sand-trap of halos and horns. It only becomes trustworthy when the hiring manager or recruiter can translate “skills used in the ABC job” into “skills needed for the XYZ job.”

Other Halo and Horns Assumptions

Making wrong-headed halo assumptions extends beyond the military.

Consider athletics. Hire me an athlete! I want everyone to have a winning attitude around here. And, by the way, hire a few cheerleaders, too. We need some pep! Halo suggests an enthusiastic athlete would also make a good employee. It’s a good thing to wake up and smell the testosterone!

Maybe.

Top athletes are select people with extraordinary physical coordination and competitive drive to win at any cost. They are not so much “team” players as they are individuals who play on a team (that old “halo” thing again).

As almost any professional athlete knows, competition for slots on any serious team is cutthroat. Would you enjoy working in a company where every associate was hired for their cutthroat competitive drive?

But don’t let the horns encourage you to turn away athletes, either. Athletic ability does not ensure job success. Job skills do.

Silly Assumptions

Halos and horns come in all shapes and sizes. I once knew a boss who would only hire people with favorable astrological charts; one who would not hire anyone who drove a VW bug to the interview; one who would not hire an applicant who parked his service truck between the white lines; and a sales manager who would not hire an applicant unless he or she asked for the job during the interview. The list goes on.

Does astrology, auto preferences, parking preferences, or interview behavior ensure job success? Nope. Job skills do.

What to Do? What to Do?

Recruiters and hiring managers are hired for their ability to make good hiring decisions. That is, hiring decisions that are correct most of the time. And they use tests all the time. People are considered (or not) based on resumes, application forms, and interviews. The problem with each of these tools is they have a hearty portion of subjective opinion, and subjective opinion is affected by halos and horns.

Professional musicians often audition behind curtains. Sound weird? This practice helps the conductor focus on the music, not the person. For all they know, the violinist behind the curtain could have one big eye in the middle of their forehead (no offense to members of the Cyclops family) or look like Jessica Alba. It’s the music that is important.

“Behind the curtain” recruiting and hiring manager tools include converting job requirements into lists of measurable competencies, using behavioral interview technology, and validated tests designed to predict job performance such as simulations, cognitive ability tests, technical knowledge tests, and so forth. Oh, yes, and be sure to follow the MTMM method.

That’s nuts! It’s too hard!

Think of it this way: you only have to follow them if you care about making an accurate hiring decision. You can always fall back on halos and horns.

Some of you might have heard or read about Google and its bio-data applicant screening process. As cited in a recent New York Times article, its basic approach is supposed to be simple:

• Survey current employees on a variety of characteristics and traits, including teamwork, biographical information, past experiences, and accomplishments.
• Statistically determine which of these many traits differentiates employee performance.
• Develop an online survey to gather intensive bio-data questionnaire.
• Score applicant responses based on the number of performance indicators each candidate possesses.

All in all, this is supposed to be a more “scientific” approach to hiring. Don’t rush for the bio-data survey solution yet. At the end of this article, you can decide for yourself whether this sounds like a good process (and be thankful you were not the one who convinced management bio-data was a good idea).

Bio-Data?

Biographical data or “bio-data” surveys are well-researched in the literature. They work on the same principle as behavioral interviewing: what was done in the past predicts what will be done in the future. The big differences between bio-data questionnaires and behavioral interviews is that a good behavioral interview is backed by a thorough job analysis, interviewers can ask follow-up and clarification questions, and multiple interviewers coordinate the information.

A bio-data form depends entirely on the people who created the items, the scoring algorithm, trained analysts who look for trends, and the specific position. More about these later.

Both bio-data questionnaires and behavioral interviews are self-reported information subject to applicant creativity and being in touch with reality. In general, they both have about the same degree of predictive accuracy. Let’s peel back the bio-data onion.

Our pet whimpers. Dr. Dolittle is on vacation so we don’t know when it started, where it hurts, or whether there are other symptoms. We just know Fluffy is in pain.

Low employee performance is similar. We can evaluate employee satisfaction, voluntary turnover, training success, or terminations. But these are all end results. They don’t actually tell us the root-cause of the performance problem.

Traditionally, performance problems can be traced back to bad management (i.e., incompetent managers, conflicting goals), unpleasant working conditions (i.e., wages, benefits, environment, and insufficient resources), lack of training, and/or poor job skills. There are many reasons why people under-perform.

Finding out the root of the problem is the most important part of developing a pre-hire test. If we don’t know the root cause of low performance, or the root cause of high performance, any hiring solution will be half-baked because it won’t address the issue.

What’s Performance?

How do we measure employees’ performance? Is it measured by judging whether a company’s growing fast? Is it measured by whether a company is profitable? There are too many environmental factors for us to assume that employees are the only growth factor.

How about performance appraisals? We all know performance appraisals suffer from the “no one here is perfect,” “everyone here is perfect,” or “forced-rank” syndrome. In addition, performance appraisals tend to be part fact and part management opinion. Basically, we can never really know what performance appraisals measure. Two managers may rate the same employee differently.

Let’s say we have objective productivity data available such as units per hour, mistakes per 1,000, cold calls per month, cross-selling revenue, or customer service surveys. These are better indicators of performance because they are less likely to be affected by other factors, but we still have to account for things that might influence them.

Did the machinery malfunction or was it newly renovated? Were mistakes suddenly calculated on a different basis? Was the territory newly acquired or was there a company promotional campaign?

You get the picture. Accurately defining performance and controlling for outside factors is absolutely critical. Otherwise, you run the risk of measuring garbage.

What Can Current Employees Tell Us?

Assuming that performance data and root-cause data are under control, let’s look at current employees.

Current employees are a great deal alike. That is, they are all “good enough” to stay hired. The differences between high- and low-performing current employees (assuming we are exceptionally clear on the definition of performance) are generally very small. So small, in fact, that performance differences might be due to pure chance (now, wouldn’t that mess up the recipe for success?). Applicants, on the other hand, are very different.

In addition to the applicant-employee difference, not all jobs have the same skill requirements. Does it come as any surprise that singing in the glee club may have nothing to do with administrative skills? I know sales managers who only hire salespeople if they played athletics in high school (the poor-man’s bio-data test). Fifty percent consistently fail within the first year. That’s no better than chance. Did the athletics bio-data question work? You do the math.

Pick up any good book on bio-data and you’ll see that trustworthy bio-data scores are exquisitely sensitive to positions. In other words, salespeople, first-line managers, and administrative support all might have completely different bio-data profiles associated with job performance (there’s that p-word, again).

High performers are usually specialized beasts who do not conform to any norm. They are usually so good that they operate on automatic pilot; or they cut corners to achieve their goal. I recall a marketing manager who stole his prior employer’s product secrets and used them to reduce development time. There’s a good high-performance role model? Right?

You may think that you should figure out what your corporate culture is, and then examine whether applicants fit that. But companies are not static. They start as small enterprises founded by highly motivated entrepreneurial folks who dine on the vending machine goodies, shave in the bathrooms, and sleep on cots.

After a while, the free-wheeling entrepreneurial environment changes into a bureaucracy, then it changes again, and so forth. Anyone who recalls the rise in dot-com businesses, or remembers how big business fares when leadership changes, knows that today’s culture-fit may not last.

I once worked for a company that hired smart, highly motivated people for plant start-ups. Two years later, the plant management complained they had “all leaders and no followers!” Be careful what you measure. You just might get it.

Statistical Sense and Nonsense

Statistics are dumb?but useful. They can tell if two numbers are correlated; but they cannot tell if one number causes the other.

This is really important if you want to develop a test that predicts job performance.

I can statistically show that blue eyes and blond hair are correlated, but we all know that blue eyes do not cause blond hair. Jan Lethen, a statistics professor at Texas A&M, cites more correlations as an example of statistical nonsense: shark attacks are correlated with ice cream sales; skirt lengths with stock prices; and cavities with vocabulary size.

When a broad sample of items are given to a broad sample of people and statistically analyzed, some correlations are inevitable. But if the items do not cause the behavior, they are bogus. They end up screening out qualified people and screening in unqualified ones.

Other problems include sample sizes. Statistics represent general trends between two groups, each of which must have the same bell-curve. Bell-curves need about 25 people at the minimum. They really work when the numbers get closer to 250.

Remember our employee-applicant difference discussion? The employee-skills bell curve would be shaped more like a finger. An applicant bell-curve would be shaped more like a ripe pimple. Comparing data with different bell-curve shapes can add major error to the numbers.

When Does Bio-Data Work?

Bio-data questionnaires provide the best results when the following criteria are met: a tight-knit group of similar jobs; a tight-knit definition of job performance; a skilled analyst interviews multiple people looking for causal bio-data items; bio-data items are administered to a large number of current employees and analyzed for performance differentiation; the test is given to a large number of applicants who are hired regardless of their scores; and after a period of adjustment, bio-data scores and job performance are statistically compared.

So, I ask. Just how predictive does the Google approach sound to you?

By the way, I don’t want to publish wrong-headed information. And I know how reporters distort facts to make a good story. So I welcome anyone from Google to post (or Todd Carlisle to address when he speaks at ERE’s San Diego conference in April) to clarify or address how they worked through these scientific questions.

Assessment! What a concept! Imagine a world where job applicants are screened for their job skills?before being hired! Wow!

Assessment = Judgment = Test = Interview = Application = Resume

It’s so simple, it’s complicated.

Folks, anyone who screens resumes or applications, conducts interviews, or reviews past work history to predict future job performance is already using assessments. Ads, postings, websites, and referrals may bring applicants to your doorstep, but assessments separate employees from wannabes.

So wake up and smell the coffee. Assessments are not weird, foreign, or unusual. They are used every time someone places an ad in a trade newspaper, posts on a job board, conducts an interview, uses a “smart” Internet application, “sells a pencil,” or gives a test.

The sooner we realize our profession is up to its armpits in the assessment swamp, the sooner it can change our image from a “learn-as-you-earn” job to one where highly skilled professionals command respect. Can anyone escape the assessment swamp? Sure: advertise everywhere and hire anyone who applies.

Sourcing as Assessments

When we post ads to a specific newspaper or website, or ask for personal referrals, we are using a source-related assessment. In other words, we expect these sources to minimize unqualified job seekers.

For example, posting on an Internet board excludes people who are not computer-savvy enough to navigate the Internet and post to the site. Posting in the WSJ excludes people who are not business-savvy. Posting to a site catering to a specific ethnic group excludes everyone who is not a member of the group.

You get the idea. Sourcing choices are a subtle form of assessment because they act as not-so-subtle pre-qualification screens.

Interviews as Assessments

A recruiter or hiring manager might think, “Hmmm. Interviews are ways to get to know someone. If I get to know you, you should be able to do the job.” Well, if our objective was to know someone at a surface level, then interviews would be appropriate.

However, anyone with a few years of career experience knows that interview skills and job skills are two entirely different things. As we all know, even the most likeable people can turn into incompetent employees.

The problems with using interviews as assessments are well-known: the interviewer often has an unclear idea of the skills required; question techniques might be leading or unclear; the applicant may not be able to recall a good response; applicants fib; personal appearance may affect the decision; and so forth.

In a desperate attempt to add credibility to the interview, people advocate wrong-headed questions like the “gotcha!” (e.g., when candidates are invited to say something negative about themselves); the “one-question-wonder” (e.g., when interviewers have a secret internal job-standard against which everyone is measured); or the “pseudo-shrink” (e.g., where the interviewer thinks he or she is a practicing psychologist and asks deep questions like, “If you were a tree, what color would you smell like?”).

Yes, interviews are assessments. They have questions, answer guides, and pass/fail standards. Once you screen out bottom-feeders, they tend to be about the same as chance at predicting performance.

Automated Assessments

Automated assessments usually include some form of computer interface that presents applicants with a situation/question/simulation/application blank, asks for a response, and scores the answer. The assessment could be located in a kiosk at the front of a store, on a computer in an office, or even presented through the Internet. Its purpose is the same as the interview: screen-out unqualified applicants.

Does it work? Maybe. It is probably better to think about how well it works. Garden-variety interviews have almost no predictive power. Structured interviews are better because they are more highly focused. Tests and simulations are among the best predictors because they are exceptionally hard to fake.

But that’s not all you need to know about assessments. You need to master the dreaded validity beast.

Before I started this profession, I was certain that motivation test scores were closely associated with intelligence. That is, I was convinced that high/low scores on motivation tests were valid (there’s that word again) predictors of intelligence. But after a few studies, I saw that 99% of the time, motivation scores turned out to be wrong. I should have known better.

Do you really know whether your test/assessment/interview predicts job performance? I don’t mean war stories; I mean solid investigations to determine whether the applicant is qualified or unqualified.

What can you do to ensure that test scores are valid? First, understand that “performance” can mean a lot of things.

What Is Performance?

Some people (professional and otherwise) get confused about performance. Is performance the number entered on performance appraisals (which everyone lies about anyway)? Is it the score at the end of a project when everyone gathers to take credit for success or assign blame for failure?

Or is it something else?

It helps to think of job performance as a fruit salad: little bits of pears, apples, peaches, pineapple, and grapes residing in a bowl. Each fruit represents a different set of skills.

Pears equal cognitive ability or the employee’s ability to analyze situations and make good decisions; apples represent the ability to learn and apply new information; peaches symbolize planning and implementation skills; and so forth. Each piece of fruit represents a job-related skill that, when used properly, contributes to overall job performance.

Most folks tend to put individual skills into a blender, then press the annihilate button. Within seconds, individual skills disappear into a performance puree. In the process, they lost the ability to examine individual pieces and have no idea whether the pears were exceptional or the peaches unsatisfactory.

A valid test requires comparing apples to apples: that is, on-the-job learning to learning-ability tests; on-the-job persuasiveness to persuasiveness simulations; on-the-job planning to planning tests; and so forth. Comparing fruit puree to 35 dimensions of personality, an industry job “norm,” or a specific test may produce data that looks and sounds credible, but it can produce nonsense numbers.

Performance is not a single dimension. It is the score at the end of the game, the baby at the end of the pregnancy, or the black eye at the end of the long argument.

Someone once argued that his assessment system was so simple, it did not require a Ph.D. to use. I suppose this person thinks this is a good thing. Hopefully, after reading this article, you will realize that “simplicity” and “highly effective” don’t play well together.

You may not need a Ph.D. to be a competent assessment professional, but if you expect your assessment/test/application form/website/interview to accurately predict job “performance,” you better think like one.

Almost every sales applicant experiences interviews and tests. Is this good or bad? It’s hard to tell. Although it would take dynamite to separate most hiring managers from their favorite test, few organizations have conducted studies showing whether its scores predict performance.

High production takes more than just “selling the pencil.” It takes a combination of skills and motivation. How can an organization use this information to identify sales winners? Let’s dissect the human element of the sales hiring process piece by piece.

Social Desirability Issues

“Social desirability” means that instead of being brutally honest, applicants say things that sound good. For example:

Interviewer (seeking potentially negative information): “What is the one thing you would like to change about yourself?”

Applicant (out-flanking the question): “Umm?I probably work too hard?and?I should take better care of my health.”

“Working hard” and “caring about health” are examples of socially desirable answers. They sound good, but only a green interviewer would take them at face value. Social desirability also affects scores on pencil and paper tests.

Sales-ability tests claim to predict sales success by asking sales-like questions, then using the answers to predict prospecting, servicing, cross-selling, and so forth. Pencil-and-paper sales tests are trustworthy tools, right? Well, think again.

Many sales tests were developed and scored based on salespeople already in the job. That can be a big problem. Why? Employed salespeople and sales applicants tend to answer differently. When the same test is later used to hire applicants, organizations are faced with a dilemma: does a high score indicate a true winner, a socially desirable winner-wannabe, or someone totally out of touch with reality?

Selective Memory Issues

Human brains are not tape recorders. Tape recorders are simple machines: what goes in, comes out. Assuming, of course, that someone remembered to install new bunny-batteries.

Hiring managers have motives and emotions that color memory and affect judgment. For example, although most sales departments are staffed with a full range of people from superstars to super-duds, most hiring managers vividly recall a time they hired an applicant who failed/passed a sales test and the employee succeeded/failed. Wrong-headed memory almost always outweighs rational judgment.

For example, I once worked with a C-level manager who was completely convinced the Watson-Glaser predicted job success. I showed him a study written by the W-G publisher showing test scores did not vary appreciably from a high-level job to a low-level job, where one or two questions could shift an applicant’s score as much as 20 percentile-points. Although we hired only applicants with high W-G scores, our salespeople still showed considerable productivity differences.

No reaction. Mr. C-Level probably had some old wadded-up WG tests stuffed into his ears. My powerful argument, backed with thoughtful studies, facts, and figures, received the same reaction as beans an hour after a BBQ. I gave up. Selective memory trumped rational argument.

Situational Judgment Tests

Sometimes people will do a thorough job of interviewing subject-matter experts and use that information to put together a collection of situations and possible responses. For example, say you are serving customers standing in line to order the newest fast-food sandwich, the HABBIE (Heart Attack on a Bun). An eager-to-die customer pushes in front of the line.

Do you: a) pretend nothing happened; b) serve two HABBIEs out of spite; 3) politely suggest someone else was ahead of them; d) take your break; or, e) beat them senseless with a Jedi light-saber from a kiddie-meal?

Sure, you say this is an easy question. But wait. Is this really a no-brainer? Is it semantics? Could a savvy applicant guess the right answer (a false positive)? Or, does the applicant have the ability to quickly learn (a false negative)?

Situational-judgment tests tend to screen out people who could have been successful if only given half a chance or who did not know the restaurant discontinued carrying light-sabers because eating them lead to an early death.

Measuring Maximum or Typical Performance?

“Maximum” performance refers to the BEST an applicant or employee can do. “Typical” performance is the average of a salesperson’s day-by-day performance. What’s the difference? Maximum performance is a function of skills (i.e., technical proficiency, intelligence, social skills, persuasive ability, and so forth); whereas, typical performance is largely a function of motivation.

Here is where it gets confusing. Overall performance equals maximum performance plus typical performance. I guess we are supposed to think that motivation is all it takes to succeed in a high-skill job. Professional athletes don’t fall off the turnip truck. They have highly tuned physical skills that set them apart. Even in an off day, a professional athlete will outperform a highly motivated average citizen.

Maximum performance requires an employee to apply his or her full complement of skills to the job. Weak skills equal weak maximum performance. Strong skills equal strong maximum performance. Typical performance, on the other hand, depends on his or her motivation to use skills. Think of typical performance as meeting the employee’s personal comfort level. Ability tests are good predictors of maximum performance.

But how do you know what you are measuring?

Situational and Behavioral Interviews

Situational interviews sound something like this, “What WOULD you do if??” They are future-oriented and almost hypothetical. A behavioral interview looks backwards. It sounds something like, “What HAVE you done?? Behavioral interviews are experience-based.

By the way, don’t make the mistake of thinking that a clever interview question is all you need to hire the best people. Accuracy depends substantially on interviewer training, job analysis data, multiple interviewers, and standardized scoring. Asking a structured interview question without this information is like giving a test without knowing what you want to measure or having an answer key.

So what kind of results do situational and behavioral interviews produce? Which interview technique is better? Which technique better predicts applicants’ maximum performance? Typical performance?

Ute-Christine Klehe and Gary Latham attempted to address these questions in an article published in volume 19(4) of Human Performance (2006). The study is quite long, but here are some highlights:

• The authors developed 16 equivalent behavioral and situational questions to assess team-player behaviors and interviewed 79 subjects using two trained interviewers using a standardized rating sheet.
• Maximum (skill-related) performance was measured by peer observations recorded during an intensive five-day work project and typical (motivation-related) performance was measured by peer observations averaged over a four-month period.

Klehe and Latham found the situational interview and the behavioral interview were both weak predictors of maximum performance. That is, neither one was very good at predicting on-the-job skills. On the other hand, both techniques were about equal predicting day-by-day average performance.

Whole Job, Whole Person

We come full circle every time. The most effective way to evaluate performance is to use measurement tools that evaluate each requirement. In addition, we need to understand that there is a big difference between what we are measuring.

Are we measuring the best an applicant can do? Are we measuring an applicant average performance? Or, are we measuring whether applicants “look like us.”

Are our tests being affected by social desirability (artificially high test scores that may have nothing to do with either maximum or typical performance)? Perhaps selective memory is affecting the tests and scores used to make hiring decisions? Or are we screening out too many qualified candidates?

Hiring is pretty straightforward when we understand it. We just have to step outside ourselves, understand what is happening, and make a concerted attempt to minimize human error.