Fake bird seed represents many vendors’ test claims … and, what users don’t know about birdseed and test validity can cost them a fortune. Test validity does not mean people like the test; or, the test has zero adverse impact; or, the EEOC approves; or, the test looks sexy. Validity means test scores consistently predict some specific aspect of job performance. For example, if high scores predict more mistakes, then low scores should predict fewer. Validity predicts on-the-job performance … both ways.

Reputable test vendors (i.e., those who follow professional test development standards) eagerly show controlled studies of test results … and, welcome questions about them. Bird seed vendors enthusiastically produce client testimonials … andget defensive when questioned. How can testimonials be unacceptable? For the same reason you cannot trust political ads. They have an agenda and are seldom supported by facts. Here is an example using a sales job:

Sales Manager Anecdote: We used XYZ test and our sales productivity increased.

OK. What is your definition of productivity? What else was happening at the time that could have affected the numbers? Did you land a big customer? Did the economy improve? Did lower and higher scores predict lower and higher sales? Are you using group results or individual data? Sales dollars are only one part of the job. What about satisfaction, service, returns, cross-selling? You see, anecdotes are rhetorical. They might sound good, but seldom tell the whole story. Anecdotes and validity are not equal. Birdseed vendors, because they don’t follow professional test-validation processes, don’t know they don’t know this.

Define Performance … or Else!

Let’s continue with our sales example. Nothing is more important than a highly productive sales staff. But wait. What does that mean? Are we discussing acquiring new customers? Farming or hunting? Cross-selling? Delivering great customer service? Customer retention? Solving service problems? Favorite golf buddies? Job turnover? Learning new products?

Get the picture? I have learned over time, especially with call centers, that many performance areas even conflict with one another. Problem Solving Quality and Calls Completed are often negatively related (i.e., it generally takes more time to better resolve problems). It drives employees crazy when an organization sets mutually conflicting objectives. So which one should they test for?

Performance is a loosey-goosey catch-all term that could actually mean something entirely different to different people. In my experience, few sales managers and even fewer HR departments ever take the time to think this through. So, before you decide on a test vendor, carefully define what you want to measure. If you think “performance” is a singular thing, then you are in a heap of trouble. If someone does not know what he/she wants to control, then any solution will be like bed wetting … warm and comfy at night, but cold and miserable in the morning.

Truth or Dare!

My bathroom scale is heartless. It tells me when I am overeating. It also tells me when I am at my healthy weight. Your hiring test should do the same thing. Good scores should have the same strong causal relationship with high performance; and, bad scores should have the same strong causal relationship with low performance. This is really important. Vendors who do not follow professional test development standards don’t seem to really understand that validation is a two-edged sword. Let’s look deeper a very common, and very wrong-headed practice.

Vendor A separates people into a good group and a bad group. The good group takes the test and the vendor averages their numbers. From that day forward, every applicant is benchmarked against the good-group average. Sound’s good? Sorry. It’s a clear sign the vendor is selling fake birdseed.

Let’s start by asking how the people were group-classified. What constitutes performance? Are good schmoozers in the same group as slow learners? How about group size? Are there enough people in a group (i.e., it takes at least 15 to 25 people before you can draw a decent conclusion). Is the bad group the same size as the good group? (Groups should always be about equal-sized.) Are the differences between groups strong or subtle? (If everyone is at least good enough to stay employed, you will probably be able to see only strong differences.)

What about the test itself? Can the vendor show proof every item in the test directly affects group performance? How strong is it? Research shows that virtually all self-reported motivation, personality, and attitude test scores have weak relationships with “hard” job skills like learning ability, problem solving, and so forth. If the test factor doesn’t strongly predict job performance, the test won’t make any difference in hiring quality … it will just make your job more difficult.

One more comment about group scores. They tell you about groups — nothing about individuals. Consider the following: people in the Top Group have scores of 20, 30, 40, 50, 60, and 70 (average = 45). The Bottom Group scores are 10, 20, 30, 40, 50, and 60 (average = 35).

So the person doing this analysis figures that producers score an average of 45 — so let’s go test people and hire the ones who score 45 or more. Whoops! If we used top-group averages as our standard, we would eliminate three top producers and hire two bottom ones. Fake birdseed alert!

Separating Pros from Pretenders

Setting hiring-test standards is an all or nothing game. There are no shortcuts. In my personal experience, wrong-headed vendors are seldom intentionally deceitful. They enthusiastically believe in their fake birdseed; after all, people who make things with their own hands seldom welcome criticism. So, they rely on client anecdotes, claiming that is sufficient proof of validity. Some will even claim that the EEOC has validated their test. Sorry. This is completely wrong-headed and foolish thinking.

If they rely on vendor claims, users will never know how many good candidates they turn away, nor how many bad ones they will hire. They always pay the price for this mistake later. You see, legal challenges seldom happen in the hiring phase. They happen on the job. Challenges begin when incompetent employees challenge termination or being overlooked for promotion. Forget the short term and six-month guarantees. Bad hiring decisions start showing themselves about a year later.

So how do you identify a pretender? Anyone who is:

• Producing client testimonials (not tightly controlled studies) claiming their test is valid;
• Getting defensive when questioned;
• Claiming their test doesn’t actually predict performance, but can be helpful;
• Claiming the EEOC has approved their test;
• Setting standards based on group or job averages;
• Focused primarily on training, not professional test development;
• Giving everyone a broad-based test (i.e., not based on performance requirements) and then measuring averages;
• Giving everyone a broad-based test (i.e., not based on performance requirements) and then measuring differences;
• Believing a self-descriptive test strongly and accurately predicts job skills;
• Not able to produce a technical manual documenting what the test measures and why that factor leads to job performance;
• Not clear on the definition of what the test actually predicts;

There are others, but this is a good start. Here is a quickie birdseed question users should ask every vendor: “Was your test specifically developed to predict job performance? If so, what part?” Any answer other than “Yes” means the test probably won’t work.

Birdseed or Not Birdseed!

As you might imagine, birdseed vendors complain the loudest. That’s really shameful. Validation principles are taught in major universities throughout the western world and religiously followed by every professional test development house. Just because a vendor does not know what they are is no excuse. It reminds me of Gary Larsen’s little fat boy trying to enter the School for the Gifted and Talented by pushing hard against a door that clearly say “pull.”

Here are some believe-it-or-not examples:

V1: Vendor (who sells a self-reported personality test) … All you care is about assessment. Don’t you care about performance?

A: Hello! Assessment is anything used to evaluate a candidate and predict performance. Besides, there is abundant literature showing self-reported tests are miserable predictors of skills like problem solving ability, planning, and teamwork. You want accuracy? Start selling tests that measure hard-to-fake applicant skills.

V2: Vendor (who sells a post-WWII NAZI atrocity test). Our test is validated. See our report. Wanna be a distributor?

A: No, thank you. I am not in the market for a concentration camp commandant. Besides, a technical report filled with anecdotes from unqualified people venturing their unsupported personal opinions about your test does not meet professional test standards.

V3: Vendor (who does group-level averaging). Group averaging is just another form of validation.

A: No. It’s not. Your test has no clear performance criteria; no proof a specific factor causes performance; group data is being used to make individual conclusions; and, your groups are so small, the numbers are either nonsense or chance.

U4: User … If I use a test, I’ll never place a candidate!

A: If there was ever a statement concerning the sad state of applicant screening, this was it!

U5: User … We like the DISC/MBTI/ACL/CPI/16PF/MMPI/Caliper test so much, we decided to use it for hiring.

A: That’s interesting. As far as I know, none of these publishers claim their test predicts job performance. Some even strongly recommend against it. Perhaps, you know something the publishers do not? Think about it. Just because a test measures a difference between people, does that mean it also predicts someone’s job performance?

U6: User … We use tests to match candidate personalities to managers.

A: That might be a good idea, unless company culture never changed; managers never changed jobs; people never changed departments; or, cloned personalities never lead to group-think.

U7: User … We interview. We don’t use tests.

A: If you ask questions and make hiring decisions based on applicant answers, how is that not a test?

V8: Vendor (after learning what it takes to meet professional test requirements) … I can’t do that!

A: That said it all.

V9: Vendor … We keep adjusting top group scores until we get the maximum individuals in the group to pass. The results become our hiring standard.

A: Fine-tuning junk science yields finely-tuned junk science.

V10: Vendor … We compare every applicant against a country-wide manager/salesperson/driver/XZY job norm.

A: So, you are assuming all jobs/companies/industries with the same title are alike; everyone in the group norm performs just like your people are expected to perform; every individual in the group norm matches the group average; jobholder answers are identical to applicant answers; applicants never try to make themselves look good on tests; and, every factor in the norm affects performance? Sure.

Q: Recruiter … My system works best. I know, because most of my placements survive the guarantee period.

A: Good for you (or, for your business, at least). From my experience, only a really bad candidate will fail to survive a guarantee period, and most organizations will go to great lengths to avoid the pain of starting another search. Besides, it generally takes about 18-24 months before an employer can separate job-learning from job-performance. Measuring success by guarantees is not the same as measuring success based on whether someone can do a job.

Q: Vendor … My system works great. It matches each candidate to a job profile.

A: Oh really? Do all jobs in the target profile perform the same work? Is every profiled-person a fully skilled, high-performing employee? Does each factor in the profile carry equal weight or are some factors more critical than others? Do individuals making up the profile actually match their own group average? Suppose there are two job groups of 100 people with the same average score. However, in one group, individual scores range from 10 to 90 … in the other group, scores range from 45 to 60 …  are the two groups really the same?

Q: Employer … Our attorneys recommend against tests. We interview candidates.

A: Interesting. What do you call it when: 1) you have something you want to measure; 2) you ask a candidate questions; and 3) you score the answers? Tests/application blanks/sourcing venues/interview questions/resume reviews and so forth = tests. And, their accuracy varies widely. By the way, corporate attorneys tend to be trained as contract experts, and labor attorneys tend to be litigation experts; so, it is really up to HR to do the front-end prevention work (scary, yes?).

Q: Recruiter … Job-fit is more important than anything else.

A: Maybe. Let’s break apart the job into pieces, starting with the critical competencies used to get the job done; then we add manager-fit (may change), department-fit (may change), organizational-fit (usually stable until the next merger/sale/acquisition), and job-fit (probably constant) … now, which “fit” category are we speaking about, and why does fit trump everything else?

Q: Vendor … Our personality tests accurately predict on-the-job behavior.

A: Hmmm. While some of the better-developed tests include scales that identify inconsistent answers, does every personality factor make the difference between job-success and job-failure? Even if you can identify specific performance factor(s), can you actually be sure candidates don’t try to fake good, attempt to match a specific job profile, present an idealized image, or report their true self? In a controlled experiment, I once compared ratings from experienced professionals (e.g., experienced people trained to observe and classify behavior) with 266 candidate personality test reports. Guess what? There was almost no correlation between test scores and behavior.

Self-descriptive personality tests are poor measures of performance. If behavior is critically important in your job (i.e., managers, salespeople, customer service, and so forth) the best way to get a trustworthy reading is to put the candidate in a position where he or she must show you what they can do.

Q: Employer: We don’t have to worry about being sued by the EEOC.

A: Not right away. Consider this. Two companies each hire 100 people. The Great-Hire Company uses a combination of validated behavioral interviews, tests, and simulations. They screen-out everyone who cannot demonstrate (i.e., show) required skills. The Know-Em-When-We-See-Em Company uses traditional interviews. They screen-out people who cannot pass an interview. Which company has the better-skilled workforce? Which has a better career path and deeper promotion pool? Which is more productive per employee? Which company is less likely to get sued for wrongful termination or not promoting protected groups? Forget about the EEOC. Start worrying about organizational bench strength.

Q: Vendors and Recruiters: You are just trying to get people to use tests/assessments.

A: They already are…I’m just trying to get them to use better ones; that is, identify more and better skilled employees.

Q: Employer … We use tests, but don’t use the scores to make hiring decisions.

A: Then don’t give your test until after the candidate is hired.

Q: Employer … we’re satisfied with our present interview and test system.

A: That either means one of two things: 1) you know precisely the cost of low performance, have done formal job studies, validated all your tests, ignore personal anecdotes, and track adverse impact at every decision point; or 2) you never calculated the cost of a bad hire. Which group do you belong to? In case you belong to the second, you might like to know that traditional interviews and unvalidated tests produce enough poor employees and managers to cost your organization anywhere from 10% to 50% of base annual salary. This number comes from recruiting, lost opportunities, turnover, training, coaching, employee mistakes, over-staffing, litigation expenses, lost salary, and so forth.

Q: Recruiter/Employer … we ask our hiring managers to define job requirements.

A: That’s a big problem. Who knows more about what it takes to do your job: you or your manager? Managers might define overall performance, but job holders know the most about what the job takes moment to moment. And, senior managers know how your job will change in the future. You think you can get all that from a hiring manager?

Q: Recruiter/Employer … if we did all that, we would never hire anyone.

A: Are you saying that if you identified critical competencies by interviewing incumbents, managers, and visionary managers, and then used validated tools to screen-out everyone who did not meet the job requirements, no one would pass? Hmmm. Please tell me how you explain all those people already doing the job?

Q: Trainer … just hire the people. We’ll train them to competency.

A: You read a book on this, right? Have you forgotten the number of times you were asked to train an incompetent person into a competent one? Or cringed when asked to link training dollars to either behavioral change or personal effectiveness? Training might enhance skills, but it seldom, if ever, changes a job-incompetent person into a job-competent one. Training is a skills enhancer, not a magic wand.

Q: Vendor/Employer/Recruiter … we use a popular intelligence test to make hires.

A: You know, of course, intelligence tests are both good and bad? The good part is smarter employees (i.e., higher scoring candidates) tend to do better than their less-smart team members. The bad part is you run the risk of: 1) filling an organization with people who are too smart for the job; 2) adversely rejecting too many members of a protected demographic group; 3) hiring people who practiced the test multiple times on multiple job interviews; 4) or restricting the size of your candidate pool. Intelligence tests are great performance predictors, but only if they pass the Three-Bears-Test: too much, too little, or just right for the job.

Q: Vendor/Employer/Recruiter … why is the intelligence test so important?

A: It is not critically important in all jobs … only jobs where the person is required to learn, solve problems, analyze information, make sound decisions, and so forth. It’s common sense really. Put a team of one dozen dull employees alongside a team of one dozen smart employees, and who do you think will do better?

Q: Vendor/Employer/Recruiter … the DOL ‘Guidelines and ‘Standards don’t apply to me.

A: Actually, they apply to everybody with a role in recruiting, evaluating, placing, training, or promoting employees/managers. If you don’t follow the ‘Guidelines and ‘Standards, then hiring and promoting fully-skilled employees will never be more than a game of chance. Test liability? That is always responsibility of the test user. EEOC? That’s the least of your worries.

Q: Vendor … my tests are approved by the EEOC and validated for all jobs.

A: And pigs can fly. The EEOC does not approve vendors. Tests have to be validated job by job, unless, after doing a job analysis, one a test user can transport (e.g., borrow) another user’s validation work.

Conclusion

This is by no means a complete list. But the logic is clear: if you don’t follow best practices, your employees and managers will range from good to bad; and, bad employees means higher turnover, more training expense, excessive recruiting costs, increased legal exposure, more people to do the same work, and wasted payroll.

If you don’t do it right, you will do it wrong. There is no alternative.

Job Failure and Job Success

My profession is studying jobs and designing tests/exercises/interviews that measure both skills and attitudes. Extensive job experience and exhaustive graduate studies have brought me into contact with hundreds of managers in large corporations. One of my first activities has always been to interview people, either in the job or supervising the job, and ask: “What are all the reasons employees succeed or fail in this job?” The following responses are typical:

Can’t manage time, Makes bad decisions, Can’t get along with people, Doesn’t seem to care, Can’t sell, Can’t lead others, Poor communicator, Not honest in dealing with people, Poor communication with customers, Poor planner, Doesn’t follow up, Can’t learn new information, Poor attitude, Doesn’t show initiative, Can’t see the forest for the trees, Doesn’t consider enough information, Never anticipates consequences, Has poor judgment, No tact, Not a “people person,” Ignores deadlines, Inflexible, Doesn’t like the work, Not a team player, Doesn’t support organizational goals, Can’t see the big picture, Can’t make a decision, Bad fit

Now that we know what people who supervise (and do) the job say, let’s look at how HR usually answers the same question:

No one helped them, Not given direction, Bad management, Never trained, Bad fit, De-motivated, Not really sure, Personality conflict …

(Yes, it’s usually a very short list.)

Notice the difference? Managers and job-holders cite about 80% skills-related items and 20% attitude-related ones. HR, on the other hand, almost always attributes performance to victimization. I think this is a pretty significant finding, don’t you? Now consider the following claims from the email vendor:

The vendor says:

• Our research shows 89% of bad hires are due to attitude such as coachability, emotional intelligence, and temperament.
• Source credibility for this data is attributed to articles in Fortune, IndustryWeek, and other newsstand magazines.
• Almost everything about job attitudes can be discovered from an interview.
• You can learn all your need to know by attending a 60-minute webinar.

On what planet?

Obviously the vendor’s body of research has been kept completely secret from the hiring-science community. Sure, if an employee arrives on the job with a full complement of skills, attitude can have a big effect. But, by completely ignoring ability, do you think this vendor is appealing to people who supervise the position, or the HR community? More to the point, if this product ignores 80% of job experts’ data, do you think their product can possibly be as good as they claim?

I’ve been on the end of many of interviews and can say with certainty most magazine authors are less-than-expert in the subject matter. In fact, they work hard to find simple sound-bite answers to complex questions, seldom caring about hard research because it makes for dry and uninteresting reading. In short, articles published in mainstream media are a better indicator of clever PR than expert peer-reviewed research. If you want opinions, visit the newsstand or bookstore. If you want facts, read unbiased hiring research studies.

Measure attitude using only an interview? Sure. For one thing, everyone knows a smart candidate can dance rings around a typical interviewer. For another, interviewers neither have the training nor the experience to be personality psychologists. Anyway, abundant literature (I know… booooring!) shows clinical evaluations (e.g. trained psychological experts) are inaccurate predictors of job success. You won’t find this information in the WSJ or HBR because it is not “catchy.”

Hard Facts

In my experience, there are thousands of training-program vendors, thousands of junk-science test vendors, and a few hundred professional selection tool vendors. Seldom will you find both training and professional selection technology coexisting — the technology and philosophy is totally different.

If a vendor’s website talks all about training, it’s a good idea to pass. You see, developing a professional hiring test takes more than drafting a few questions. It takes months of editing and statistical analysis to demonstrate it actually predicts some aspect of job performance. BTW: This would be a good time to revisit the manager’s succeed-or-fail list.

If you are responsible for making hiring decisions, be careful of self-promoting vendors entering the hiring and selection marketplace. Professionally, I never found one sufficiently qualified in the science of test development to develop a product that will eliminate dead-wood candidates. And believe me, if you use junk-science tests, you will learn the hard way they don’t work as advertised. Furthermore, all that dead-wood will be on your payroll.

This warning is true for all products that suggest they can teach you to analyze a candidate’s motivations, use self-reported personality surveys to match performance with a data base of job titles, predict job performance without actually measuring skills, or use any other method that is less than comprehensive or validated. From a legal perspective, the user, not the vendor, is always responsible for test use.

I have been accused by some of promoting “assessments?” Get real. It’s semantics: Interviews, resume reviews, application blanks, surveys, tests, sourcing, and so forth, all all assessments. Assessment is just another word for test, and, valid tests are useful tools for evaluating qualifications. If you don’t have proof your test/interview/assessment predicts job performance for your job in your organization, then you will assuredly turn away good people and hire useless ones.

Why should you worry? Experts estimate poor employment decisions cost about six month’s salary, not to mention perpetuating HR’s professional reputation for quick, ineffective solutions to complex problems. Forget vendor hype. Simple, one-step hiring solutions are nonsense. They don’t deliver.

Professional recruiters are citing an increasing number of independent studies claiming there is no difference in employee quality between internal and external recruiters; so, they argue, why should organizations hire full-time internal recruiters when external ones deliver the same results … cheaper? If I were an executive looking for ways to reduce costs, that argument would resonate with me. So, if you have anything to do with recruiting in your organization, how you react to this crisis could make a big difference to your career.

Same Old Same Old

Recruiters (both inside and outside) are like frogs swimming in a pot of cold water. Experiencing slowly rising temperatures, they are totally unaware they are about to be cooked. (Actually, I never boiled a frog, so I’m taking this story at face value). In fact, the last recruiting conference I attended was utterly packed with sourcing candidates and noticeably shy on evaluating them. When I buttonholed both recruiters and sourcers about the importance of qualifying candidates, their eyes would literally glaze over. They either knew it all … or cared less. I emphasize this story because I have not yet met a line manager who thinks recruiting is doing a good job qualifying candidates. And, guess who controls the money?

Aside from sourcing, the traditional method of hiring is to screen a pile of resumes, run applicants through interviews, and do background checks. We all know it’s easy to fake interviews, and results are mostly personal opinion. Furthermore, you don’t need research to know about half of new hires fail to meet expectations. Just look around. Is it any wonder HR outsourcing is a growing industry?

Nuts and Bolts

I did not invent best-practice hiring tools. They evolved from many years of research that, in my experience at least, most recruiters blow off as being too much work. Best practice starts with knowing critical skills associated with each job, then measuring them with hard-to-fake behavioral interviews, tests, simulations, and exercises. Does this process ensure 100% perfect hires? That would be nice, but no. There are simply too many factors that affect the future. Best practices significantly reduce the number of hiring mistakes. However, one fewer hiring mistake means one additional highly productive employee. Put another way, we know in a typical organization that 20% of the people produce 80% of the results. So, imagine what it would be like if that number was reversed to where 80% of the people were top-notch.

The reasons for poor performance are seldom the employee’s problem. He or she was coached to say anything to get a job; job competencies were unclear; and interviews were easy to fake. Imagine that!

Best practice hiring tools are different: they are considerably more accurate than traditional interviews, highly focused, and hard to fake. If you want management to consider recruiting or HR an invaluable department, I suggest ignoring job titles, organizing jobs into families (i.e., jobs with similar competencies), studying each family to identify job-critical competencies (i.e., ones that can be measured), developing reliable and trustworthy measurement tools, setting professional cut-points, and training your people how to use them.

Still need to use a professional recruiter from time to time? Professional recruiters usually have access to impressive networks or are able to screen high volumes of candidates. Let them know, however, that you will require each submitted candidate to successfully pass your best-practice screen. Don’t be surprised of instead of the usual 2-3 candidates, you have to test about seven to find the right-skilled person. Will the recruiters complain? Probably. But they don’t have to live with the consequences of a bad hiring decision.

Piece of Cake…Not!

You might not have the expertise in-house to set up a best-practice system. In that case, consider hiring a psychometric expert to get you started. I’m not referring to freelance salespeople who market out-of-the-box tests. They are probably well-intentioned, but limited in their range of tools, and totally unaware of limitations (e.g., if the only tool you know how to use is a hammer, every problem looks like a nail). Professionals can be identified by career and academic credentials, professional memberships, time spent interviewing people doing the job, use of different tools, professional validation processes, documentation, knowledge of the DOL Uniform Guidelines, and use of tests specifically developed for predicting job performance.

The entire investment of a best-practice hiring system is often recovered in 60-90 days. Isn’t it worth it to cut-through the sales-pitch and get the employee you actually thought you were getting?

Dead or Alive?

Is HR dead? That depends. Keep up the same-old practices, and the answer is probably, “Yes.” Move into the 21st century and master best practices, and I predict you will become managements’ new BFF.

Briefly, it helps to think of organizations this way: you can hire and promote 100 people whom 20% are high-quality, 20% low-quality, and the rest so-so; or, you can hire and promote 100 people, of whom 90% are top-notch. The first situation is the norm. It’s what you get when obstacles get in your way. The second option is the exception.

It’s what you get when your obstacles are controlled and minimized.

#1 Overemphasizing Past Performance

Past performance is often used as a predictor or even a gateway for hiring or promotion decisions. Of course we all expect a candidate to do well in a past job; but, if the requirements of the new job differ, we have to use an additional set of tools to measure whether the candidate has additional skills. This is almost always the case when hiring someone for a new job; screening college candidates with no prior job experience; promoting individual contributors to management; promoting managers to executives; making job reassignments; and, so forth.

You can trust past performance for skills that transfer, but you’ll need new assessments to evaluate new skills.

#2 Lacking a Common Denominator

Organizations seldom have a reference point for comparing job skills (i.e., requirements) to human skills (i.e., KSAs). Recall the problems associated with a past space shuttle mission when European engineers used the metric system and U.S. engineers used the English system? Organizations must learn how to clearly define for each job the criteria associated with business necessity and job requirements (i.e., how it is to be done). “How” data serves as a bridge to candidate skills. Without a common skills denominator (i.e., a language of how to get from point A to point B) mistakes will be the norm.

It’s bad mojo when job requirements are written in the metric system and candidate skills use the English system.

#3 Focusing on “What” Instead of “How”

Recall the Wizard of Oz. The story is not about the Oz. It’s about how the characters got there. The same is true of performance. If you measure people by performance alone, you encourage using any means to achieve the ends. Results are always affected by being at the right (or wrong) place at the right (or wrong) time. This makes informed decisions a leap of faith. Good hiring and promotion depends on isolating and evaluating each candidate’s individual skills.

Performance is less about “what” was achieved and more about ”how” it was accomplished.

#4 Assuming People are Plastic

We have all seen newly hired and promoted employees turn out bad. We have also seen the difficulty of turning incompetent people into competent ones. It’s not news. People resist any and all personal change but expect others to be flexible. Organizations that try to make someone smarter, change their motivations, make them more sensitive, and so forth, will spend money but get little or no return from their investment. Can someone with a slight amount of job talent, be made better? Maybe. Can someone with no talent be transformed into a fully competent employee? Good luck with that.

If you want to climb trees, it’s easier to hire a squirrel than send a chicken to a workshop.

#5 Management Interference

Not everyone buys into rigorous selection systems. Many managers tend to think hiring expertise is conveyed upon promotion and strongly argue in favor of gut decisions. I emphasize almost every hiring or promotion decision is gut-based; the only real question is whether your gut is informed or uninformed. Professional hiring or promotion systems feed managers’ guts with objective, trustworthy, and reliable data.

An uninformed gut feeling is indistinguishable from eating bad seafood.

#6 Underestimating Financial Impact

Every below-average employee has a cost. It might be measured in turnover, mistakes, or low productivity. Professionals estimate costs of poor performance to be about 10% of annual payroll for unskilled jobs, 20-30% for semi-skilled jobs, and 40-50% for skilled and managerial jobs. Put another way, an organization with annual payroll as small as $500,000 could be squandering $50,000 to $250,000 per year. Imagine the sales necessary to offset that number! Better yet, imagine the effect on the bottom line by doing a better job hiring and promoting skilled employees.

Hiring and promotion practices won’t get better until they have a dollar value.

#7 Ignoring the Odds

Success in hiring depends more on reducing the odds of making a bad decision than it does increasing the odds of hiring a super star. The future is filled with uncertainties such as family problems, economic factors, and so forth. We can’t ever make perfect predictions. We can, however, make very sure each candidate has the right skills for the job. This is the odds game … focus on making the fewest mistakes, and the successes will take care of themselves.

It’s easier to win more often if you concentrate on making fewer mistakes.

photo by Harry Wood

Leaky tests pass-through marginal performers and, depending on the type of job (unskilled, semi-skilled, professional, managerial) they can cost organizations between 10% and 50% of annual payroll. In other words leaky hiring tests can be the single most expensive mistake organizations can make.

Here are some common-sense guidelines to dry-up leaky tests.

Self-Reported Data Leaks

A leaky hiring test often begins by asking employees to answer items describing him or herself. It might be given to your own employees or to people around the country with the same job title. Scores are collected, averaged, and used to screen job candidates. Sounds good, right? Wrong.

A couple things happen when we are asked to describe ourselves. In the best case, scores are idealized self-presentations … how we want people to see us. In other cases responders might be completely out of touch with reality or just faking it. Even when tests include an internal truthfulness scale to flag inconsistent answers, self-reported information exists purely in the mind of the candidate.

Averaging Leaks

Averaging scores is a bad thing. Averages can describe groups, but they cannot describe individuals. For example, you might believe Californians are flaky and southerners are rednecks. But when you get to know an individual Californian or Georgian as a human being, you usually learn he or she does not match the average stereotype. Accordingly, when people are assigned to performance groups, it’s rare for any individual to match their group average. Group test scores and individual scores are two entirely different things.

Score-Setting Leaks

Passing scores are supposed to predict good performance. Failing scores are supposed to predict bad performance. If you set cut-scores based on averages of high-performing groups, you learn nothing about the low group. In fact, both groups may have a lot in common! Hiring accuracy depends on knowing what makes them different.

Cause or Pause Leaks

Not all test factors cause performance. For one thing, many tests like the MBTI, the DISC, and others were all developed to measure on aspects of normal personality. They might work in a communication workshop, but not all normal personality factors apply to jobs. Research shows only three factors correlate with job performance and six to job fit. The rest are either irrelevant or overlapping.

Be wary of tests that ask a few questions and use the answers to comprehensively describe behavior. Watch out for tests that try to describe every aspect of human personality. Finally, avoid like the plague any test that comes without objective supporting data showing factor scores directly lead to (or somehow affects) job performance. If it does not cause, you need to pause.

Assumption Leaks

How many people do you know claim they are intelligent, but aren’t? Claim they are good with people, but aren’t? Claim they are organized, but aren’t? Research shows there is almost no correlation between scores on a personality test and skills such as building interpersonal relationships, solving problems, or ability to learn. Organizations that trust personality scores to predict actual skills are pre-destined to make mistakes.

Purple Dinosaur Leaks

I love you/ you love me/we’re a happy family/ with a great big hug and a kiss from me to you /won’t you say you love me too? … Ratings are like Barney the Dinosaur relationships: grouping is often based on who the manager likes best. Whenever you ask managers to rate employees’ performance, the scores will probably underemphasize actual job skills and overemphasize sociability.

Performance Confusion Leaks

Even assuming your managers are blunt-force honest, and ratings are made on jobs where numbers can be tracked (e.g., customer service, production, sales, and so forth), what happens when an employee such as a Customer Service Representative is rated both on quality and number of customers served (i.e., the two are usually inversely related)? If management cannot decide what’s most important for an employee to accomplish, then what exactly are you supposed to measure?

Employee Similarity Leaks

All those high and low performers you are studying belong to a special group: sufficiently skilled to remain employed. In other words, there really may not be enough difference between one employee and the next to measure a useful difference in scores. If you have no identifiable and trustworthy standards for comparison, then how can you set scores?

Same but Different Leaks

Some vendors offer norms for drivers, salespeople, emergency medical technicians, and so forth, implying they can be used for selection. But look at your own workforce. Are all your employees with the same title high performers, do their jobs require identical tasks, do individuals match the group average, and do high and low performers have score differences? Playing the job-norm card is an effective way to market a leaky test but it does not help the test user make better hiring decisions.

Conclusion

Leaky tests are great examples of junk science. I advise my clients to take them with a grain of salt. Leaks come from many sources: restricted score range, conflicting metrics, useless test factors, self-report errors, overemphasizing manager bias, underestimating job skills, trusting personality to predict actual skills, comparing individuals to group averages, and assuming job titles all involve the same skills. Considering the cost of water these days, don’t you think it’s a good idea to tighten-up the faucets?

Cause? What Cause?

Here is an example of traits often found in personality tests: dominance, compliance, extraversion, judgment, sensitivity, curiosity, conscientiousness, humility, and determination. First, we’ll show you a silly-science example: 1) divide producers into groups (e.g., high and low performers); 2) give both groups the same personality test; 3) see which scores differ; and finally, 4) use candidate scores to predict group membership.

After impressive number-crunching, suppose the A-list group had higher average dominance, compliance, and extraversion scores; the B-list group had higher average curiosity, conscientiousness, and determination; and, both had the same average judgment, humility, and sensitivity scores. Is this enough evidence to use the results for selection or promotion? Noooo.

Lies, Damn Lies, and Statistics

Anyone can compare two sets of numbers and tell you whether they correlate; but, it takes careful study to know whether A actually leads to B.

For example, skirts and stock markets tend to move up and down together, beach ice cream sales and shark attacks tend to move together, and watermelon sales and temperature move together. But, skirts do not cause the market to change, sharks do not buy ice cream, and selling watermelon does not cause it to be hot. You can probably think of many others, but the most important statistical concept is, “If it does not cause, you need to pause!”

True professionals know beforehand the factors they want to measure. Then, they use stats to compare scores with performance to try to prove themselves wrong! I know it does not make sense, but remember that since the future is murky and uncertain, it’s better to reduce mistakes than seek perfection. Explaining things after the fact is creative story telling. Professionals make an informed prediction, collect data, and try to disprove it.

Screwy Thinking

Returning now to our example: We already discussed why throwing things against the wall to see what sticks is unprofessional. Now let’s consider the Lake Woebegon effect; that is, the men are all strong, the women are all pretty, and the children are above average.

Let’s suppose in our previous example that shoe size was one of our factors. We know individual shoe sizes in both Group A and Group B ranged from size 6 to 12. However, Group A folks averaged size 8 and Group B averaged size 10. Does that mean an applicant wearing a size 9 will become a member of Group A? A size 12 a member of Group B? Nope. And, Nope. Group-level data tells us about groups, not about individuals! Bad analyst! Bad!

How about this? There are four people in Group A and 10 people in Group B. Aside from the problems we already discussed, can we compare the two groups? ‘Nope again. One person in Group A has a 25% impact on the group’s overall score, while one person in Group B has only a 10% impact. Furthermore, the group sizes are so small it would be silly to think scores would generalize to all candidates. It takes at least 25 (preferably, hundreds) of subjects to draw reasonable conclusions. No soup for you, analyst!

Oh, yes, one more thing. Can we trust someone with a high score in the judgment trait to be smart? Get real! Most studies show less than a 1% relationship between personality scores and cognitive skills and about a 10% relationship with interpersonal behaviors. Why? When people take a self-descriptive test you never know if they are honest, trying to make a good impression, delusional, clueless, and so forth. If you need someone who is smart, give them a problem to solve … not tell you about it!

Importance of Being Wrong Less Often

It may sound counterintuitive, but it is easier to reduce the number of bad hires than it is to find superstars. The future is murky, filled with unpredictable events that elude even Karnak the fortune teller. Folks who expect 100% hiring and promotion accuracy are going to be frustrated. No system known to humankind can perfectly predict the future. There are just too many uncontrollable variables.

The present, however, is more tangible. So, instead of trying to ensure perfect success, it’s actually easier to reduce test error by screening out unskilled people. It goes without saying that manager-employee compatibility is very important; but, in addition to personality factors, organizations expect employees to have cognitive abilities, motivations, and so forth. This is the 20% that delivers 80% of job results.

Summary

Selecting or promoting people based on silly science is more than just bad practice. It’s unethical, irresponsible, and unprofessional. Qualified people are rejected, unqualified ones are hired or promoted, and the inevitable potential of legal action increases. And, it could get worse. What do you think will happen when all those incompetent employees think they should be promoted to management?

Reducing the odds of making a wrong decision requires tests, interview questions, application blanks, and so forth that are grounded in a solid theory of job performance; that is, they measure things that cause high or low performance. If you cannot, for certain, prove you are measuring factors that cause performance, you will never graduate from the half-wrong club.

“If it does not cause, you need to pause!”

Rungs of Expertise

Expertise is ladder-like. The first rung is a pair of hands; i.e., people who make a living doing what the client asks. Usually they have some practical experience with the subject (e.g., they know slightly more than their clients); but, they are actually just skilled individual contributors. You might think of them as knowing how to use the most common Word features.

The next type of consultant is a facilitator. He or she is not necessarily more knowledgeable than a pair of hands, but knows how to manage groups. Facilitators usually start life as a pair of hands but learn little more from every client engagement. Eventually they learn to help people solve their own problems and keep them on track through group processes.

This next rung is where the herd really thins out. This is where we find people who are subject-matter experts. Not only do they have all the practical experience possible, they understand the theory that supports it. You could think of them as knowing how to use all the functions of Word as well as being able to teach others how to use any function.

Lastly, we have the experts who not only know all the functions of Word, but they can tell you what’s working, what’s not, and why. Given the opportunity, they are capable of actually redesigning the software to make it more useful and efficient. These folks are few and far between. The developer of Internet protocol and the people who developed the concept of integrated office software belong to this group.

As a side note, clients should know clearly when they need a pair of hands or facilitator who can help them move from A to B; or, an expert who can move them from A to Z. For example, people who develop their own selection or competency systems often become very upset when a subject matter expert suggests major changes. This is unfortunate because in most cases a wrongly-designed system will inevitably fail within a few years. There is really only one best-practice way to select managers or individual contributor; like it or not, the rest are flawed in one way or another.

Careers, Degrees, and Experience

Moving up the expertise ladder involves a combination of practical experience (i.e., actually performing the work), mastering its theory and technical aspects, and working with a variety of clients to broaden exposure to multiple situations. In my profession, for example, one has to be thoroughly proficient in job analysis, competency measurement, ADA, validation, multi-trait-multi-method measurement practices, 1978 DOL selection guidelines, and APA test-development protocol. If someone does not master these basics, they are not qualified to do the work.

Another indication of a non-expert is when someone tries to enter the field with an unrelated degree. For example, all test developers are expected to complete graduate courses in test design, job analysis, assessment, validation, experimental design, statistics, and so forth. The objective of these programs is to minimize error and maximize accuracy. If your consultant’s qualifications and experience are limited to recruiting, training, or an SPHR designation, their expertise is probably limited to being a pair of hands or facilitator. Don’t misunderstand. There is nothing wrong with hiring a pair of hands or facilitator if that is all you need; but, if you want to test candidates for job skills, build an integrated competency-based system that actually works, or head off legal challenges at the pass, it is not enough.

People are also usually unaware that it takes more than a psychology degree to be a qualified job psychometrician. Out of a 72-hour graduate program, for example, industrial, clinical, and counseling psychologists have only about 25% of the courses in common. The other 75% are specialized. So, while psychologists might have the same letters on their diploma, counseling and clinical psychologists are trained to help people function in society, not predict job skills for the workplace. If your hiring expert provides you with a report that looks like a mental health evaluation, it’s a clue you are dealing with the wrong kind of psychologist. Even within my field (i.e., the practical application of psychological principles to solve business problems) only a small percentage of graduates are true test experts.

As someone who both worked in the business school of a large urban university and earned two business degrees, I have also found that many business professors are better at theory than application (e.g., with the notable exception of accounting and computer science). For example, while my management professors often treated the MBTI, Hawthorne studies, and Maslow as sacred cows, my psychology professors conducted controlled studies and looked for proof. It was an eye-opening experience to read study after study debunking many business theories I thought were rock-solid.

As a case in point, you might recall Professor Mike Hammer’s popular book on redesigning jobs. Hammer’s career expertise and education included engineering and computer science. His co-author, Jim Champy, was an engineer and lawyer. It does not take a rocket scientist to realize that when jobs change, so do the skills necessary to perform them, yet these authors only devoted a page or two selecting people who could do the job (e.g., a field industrial psychologists spent the last 100 years studying). If you share Hammer and Champy’s assumptions that people are sufficiently plastic to do almost any job; ask a few bankers how much time and effort they invest getting tellers to cross-sell, or how many successful individual contributors you know who are successful managers.

Blogger Alert!

Bloggers are cropping up everywhere with opinions that are generally poorly informed. For example, one person commented on an article I wrote some time ago inferring they said as much the same thing in a presentation. I had never heard of the blogger and was curious about their qualifications to make this statement. I did a little background research and within about five minutes learned the blogger’s competency qualifications were limited to working in HR and being a trainer. Sorry, folks, being trainer or working in an HR department does not qualify one to be an expert. Be wary of bloggers who have an ALL CAPS opinion, but lower-case expertise. That’s why you will never see my name on an article about organizational development, reengineering, recruiting, compensation, or neurosurgery.

In Conclusion

I would like to offer a few thoughts I have learned along the way: one cannot identify another technical expert unless they already are one (that means we are all pretty dumb about a lot of things); the more expert one becomes, the more they realize what they don’t know (yep, it’s OK to feel really stupid most of the time); an expert will have a combination of advanced academic education (to master theory) and practical experience ( to separate theory from reality); experts belong to professional associations where expertise is a condition of membership; an expert in one subject is usually painfully ignorant of another; real experts are the first to admit they don’t know much; the more certain someone is about his or her expertise, the dumber they usually are; experts can always produce legitimate 3rd-party proof of their claims; strong (or loud) opinions are no indication of expertise; everyone has an opinion about something; and, be VERY wary of the person who does not know what he or she does not know (sometimes you need an expert to know an expert).

Over the years, I have been a salesperson, sales manager, sales trainer, and sales assessor. In other words, I have sold, managed, developed, and formally evaluated people who make a living in the strange world of selling. While filling these roles, I kept going back to grad school to learn the difference between what I was doing and what I should be doing. Along the way, I was forced to put aside many wrong-headed ideas while I learned new ways to look at the sales job.

Discovering information is key to consultative selling. Consultative salespeople acquire new business, expand existing business, and develop long-term productive relationships through learning, and people don’t learn while they are talking. So, if you are looking for the world’s best “sell ‘em while you got ‘em” sales pitch, this is not the article for you. However, if you want to hire people who can deliver long-term business relationships, read on.

Cone of Trust

Maxwell Smart (Don Adams’s character in the TV program ‘Get Smart’) used to invoke the Cone of Silence with Chief whenever he wanted to discuss secret stuff. Although Max’s Cone never worked as planned, I like the concept so well that I invented the term “Cone of Trust”; or, the special place where salesperson and prospect share a mutually productive environment. Imagine that!

Think about it. Most people like to buy things (e.g., just look at the success of capitalistic economies). But, they hate to be sold (e.g., consider the reputation of auto salespeople). The key difference between “just selling” and relationship-based selling is pitching or wooing. Pitching is battling someone into submission by overcoming objections. Wooing is building a Cone of Trust where problems and concerns are shared. Unlike pitching, wooing uses only the assumptive close technique.

A Cone…Yet More than a Cone

It goes without saying that if trust is the bread of selling, then questions are the butter (I made that up, actually!). People normally don’t like being told they have a problem. It makes them all huffy and irritated. A great consultative salesperson manages the conversation so both prospect and salesperson agree about a problem at the same time. This is much different from sales programs that recommend imitating techniques borrowed from “The Art of War.” I suppose imitation (i.e., behavior modeling) is better than nothing, but I have found highly successful people don’t imitate someone else’s behavior. They modify their own to fit the situation.

Let’s look at the conscious competency model. I don’t know where people got the idea that conscious competency means operating on automatic pilot, unaware and clueless. In my world as a psychometrician, consciousness=awareness and competency=skilled. Translated: a consciously competent salesperson is someone who is both good and can explain why. On the other hand, an unconsciously competent salesperson may be good, but if asked, cannot explain it. Consciously competent salespeople who can clarify and explain the wooing process to their staff make better managers than ones who can only suggest imitation. They keep the best interest of the prospect foremost, and while they may not push for a sale today, their strong personal relationships will almost always lead to better opportunities tomorrow. Trust, and discovery you see, depends on knowing when to discover information and when to back off.

Real-Worldly

Mind-blowing movies like Inception and the Matrix trilogy grab our attention by taking us to a place where reality and imagination blur together. Pre-hire sales simulations do that for the candidate. A well-designed sales simulation is not like a practice role-play in a workshop. It is a tightly controlled experience where the candidate has to leave his or her learned experience behind and demonstrate innate talent. For example, if we were looking for a salesperson to sell chickens we would not use a barnyard simulation; instead, we would put candidates in situations where they would have to sell an unfamiliar product. In an unfamiliar world, all their prior Rhode Island Red experience would be useless. We would observe their core sales skills.

Now, what happens on the skilled role-player side?

Not everyone can be a skilled role-player (SRP). Some people just don’t get it and others insist on bringing their personal agendas to the table. Bad dog! The skilled role-player (SRP) actually becomes a controlled part of the exercise whose purpose is to provide stimuli that invite a response from the candidate. For example, an SRP acting the part of a prospect might complain about a prior experience the candidate is unprepared for. That is the stimulus. Now, we listen for the response. Did the candidate blow it off, empathize, change the subject, try to learn more, and so forth? These are all responses that indicate whether he or she has the discovery skills to be a top salesperson.

We don’t expect a candidate to close a sale in the simulation. That rarely happens in real life, either. We expect to hear key behaviors that advance the sale, build trust, search for more information, and so forth. In short, we peel-away the veneer of well-rehearsed replies and go to the heart of selling: learning about the prospect’s situation.

Learning About Learning

Before you suggest such-and-such a written test that you believe predicts sales performance, I‘d like to point out that a simulation is a test as well. Unlike written tests, simulations are almost impossible to fake well and realistically represent the kind of environment faced by a consultative salesperson. If the candidate cannot perform in the simulation, it’s unlikely he or she will do a good job in the field. The same is true for simulations measuring a manager’s coaching ability, customer-service skills, and so forth. If interpersonal skills are important to doing the job, then you better see them in action before making a hiring offer.

I once asked a group if anyone would hire a commercial pilot without asking him or her fly a simulator. A few recruiters quickly raised their hands. The operational people in the room looked at them and just rolled their eyes skyward as if waiting for a plane to fall out of the sky. If you want to know whether your candidate has the critical skills to do the job, then accurately measure them pre-hire.

Fundamental Sales Abilities

Put on your customer hat. Do you enjoy listening to a salesperson blab? Feel like you are in a verbal contest with someone whose only objective is to get your money? Get frustrated when a salesperson does not take the time to understand your situation? These are symptoms of poor sales hiring practices.

Salespeople need four key skills to be effective: 1) trust-building skills to build and maintain customer/prospect relationships; 2) questioning skills to dig for customer problem areas; 3) presentation skills to emphasize the benefits of buying a product or service; and, 4) the motivation to face rejection again and again and again. These skills are exceptionally hard to develop. In my experience, if you don’t measure them pre-hire, you are pre-destined to live with the consequences post-hire.

In this series of articles, I’ll cover why each of these elements is key to effective selling, and why each needs to be evaluated separately.

Sales and Hiring Psychology

Thinks of a sales interview as a mixed-role mini-sales call. There is a prospect, a salesperson, and a product. The hiring manager tries to sell the company while learning as much as he or she can about the candidate. The candidate tries to sell his or her skills while learning as much as he or she can about the company. They are both motivated to hide negative information. The ultimate goal is employment. Usually, the candidate and manager openly share friendly personal experiences. They recount sales experiences and demonstrate FAB (features-advantage-benefit) by making a sales pitch. At the end of the interview, the hiring manager usually expects the applicant to ask for the job. Unfortunately, as many experienced sales managers can attest, this process is no guarantee of future success — and 80% of the salespeople continue to produce 20% of the sales.

The reasons for the 80/20 rule is many key skills are unmeasured in the sales call — ones that will later come back to haunt the hiring manager. First, the candidate and hiring manager already know why they are meeting; second, the candidate does not have to create “heat” because the manager already feels the need for more sales; third, the candidate does not have to probe and question to identify the need to hire another salesperson; and finally, a hiring offer depends primarily on whether the candidate meets the manager’s superstitious “know ‘em when you see ‘em” test.

Whether a candidate knows it or not, he or she already holds the high ground. Basic trust and need are clear when they agree to meet. All it takes for the candidate to close the loop is schmoozing the hiring manager.

Sales and Consumer Psychology

In the real world, people instinctively distrust strangers. Prospects especially distrust salespeople, products, and organizations. If prospects knew they had a problem, they would have already fixed it; if they have not fixed a problem they don’t think it’s worth fixing, can’t decide the best solution, or are afraid to make a decision. In other words, real-world prospects might just as well start the conversation with, “I don’t know you. I don’t know your company. I don’t have any problems. Why do you want to talk to me?”

Consider these statements: “I don’t know you. I don’t know your company”= I don’t trust you; “I don’t have any problems” = I don’t need anything you have to sell; “What is it you want to sell me?” = even if I had a need, I don’t see how your product or service can help. Overcoming this inherent obstacle rests on the salesperson’s ability to develop and maintain enough mutual trust to ask probing questions. If everything goes right, the questions will lead to a mutual “By golly, there is a problem here worth solving!” agreement.

Trust and Communication

Let’s look at this psychologically. Communication is a multi-level process; Level 1=casual chit-chat or party chatter; Level 2 =sharing thoughts and ideas; Level 3 = sharing feelings, fears, and concerns. If people don’t cozy-up during chit-chat, it’s not likely they will move to the next level and start to share thoughts and ideas. If they don’t feel comfortable sharing thoughts, they probably won’t move to the level where they share feelings. If a salesperson wants to get the prospect to feel the heat or sell the sizzle, he or she has to get to the level where feelings, fears and concerns are shared.

Now, here’s the rub. Suppose a prospect and salesperson get past Level 1 and are comfortable in Level 2. Suddenly, the salesperson sees an opportunity to get the prospect to “feel the heat” and decides to jump to Level 3.

How do you think a typical prospect will react when they are asked to discuss sensitive issues? Look at the communication circle: No rapport=no trust (NT); no trust=defensive prospect (DP); defensive prospect=no discovery questions (NDQ); no discovery questions = off-putting premature information syndrome (PIS-Off); off-putting premature information syndrome = no trust (NT).

Evaluating Trust

Sales applicants’ trust-building skills can be evaluated several ways: by interviewing the candidates’ past clients and customers (so-so); asking for behavioral examples (better); and, using tightly controlled one-on-one simulations that present multiple opportunities for the candidate to demonstrate these skills in real-time (highly accurate). I don’t care for–out-of-the box sales tests because I never know if passing scores are truthful or faked (i.e., salespeople are crafty little folks). Furthermore, experienced salespeople are generally more effective managing their personal image than they are building trust and discovering information with strangers.

I think an old story says it all.

A salesperson approaches his sales manager complaining the product has problems, the price is too high, the competition is too tough, and prospects keep cancelling appointments. The sales manager listens patiently until the salesperson is finished, and then says, “Let me see if I understand. You want a perfect product with the lowest price. It should be better than the competition and prospects should always roll-out the welcome mat for you?” The salesperson replies excitedly, “Yes! Yes! That’s it! You Understand!” The manager pauses, then looks the salesperson in the eye and says, “Then what would I need you for?”

That said, in case you might have missed Hiring the Kind of Salespeople You Only Dream About, I found John Zappe’s EEOC article a great companion. That is, if your organization routinely uses credit checking when hiring salespeople, you might want to know how the present Washington administration treats employers who don’t do their hiring homework.

(It’s OK, I’ll wait here while you catch up).

Get Ready!

We live in a confusing world where politicians are always on the lookout for reasons why their voting block is not being hired; where the government does not require organizations to hire unqualified employees; and, where adverse impact is not illegal. On the other hand, if someone thinks your organization does not have enough employees of the right color, gender, age, religion, and so forth, government agencies are empowered to be in your face!

Let’s start with an oversimplified explanation of the audit process. First, they (EEOC or OFCCP) process a complaint. Second, auditors use stats to examine your employee demographics. Third, if the stats show adverse impact, you are (on the face of the data) guilty of discrimination. The government could care less about how individual employees perform. Analysis is done at the group level. This can catch even the largest organizations flat-footed. Predictably, teams of $750/hour attorneys will be hired, everyone will argue back and forth for months, and eventually the organization $ettle$ out of court. (Note, although there are hiring exemptions extended to small business and special interest groups, best practices work for everyone).

Get Set!

The outcome of a legal challenge is unpredictable. The cost of a legal challenge is not. There are ways organizations can substantially strengthen their defense; and, the best part is, the government even tells you how to do it! Let’s say that, on the face of statistical analysis, your organization looks like a socially bankrupt, adverse-impact loving, discrimination-monger. No problem. Just show them records outlining: business necessity, job-relatedness, validation, documentation, tracking, and efforts to reduce adverse impact.

What? You don’t have them? You have a better way? Who made that decision? No matter. You’re screwed and your attorney is about to get a brand new Mercedes. You see, organizations that do not care about following best practices because their goal was filling open slots and surviving probationary periods inevitably have both weak employee bench-strength and shoddy legal credibility.

Go!

Let’s re-visit credit checking. The main reason why organizations only hire people with good credit scores is “everyone knows” they perform better. Right? Wrong? Maybe? Consider this: Low-income people usually have poor credit scores. High income people usually have better credit. But wait! Protected groups are usually low income. Knock, knock. Who is there? Audi! Audi who? Audi-tor! Open your wallet, and stop your clocks. This is going to hurt!

Unless you can document (using pencil and paper) how bad credit is directly related to job performance, you really don’t have a legal credibility (or job performance) leg to stand on. It makes as much sense as reading horoscopes and refusing to hire anyone other than a Gemini because you think he or she will give you twice the productivity.

Establishing Validity

You cannot just ask a credit report or test vendor for assurance his or her test works as promised. That’s a non-starter. Even if someone else already did all the validation work, you still have the responsibility to show your job is essentially the same as theirs … business necessity, job relatedness, and validity.

Making a strong case for credit checking as a hiring tool always starts with thoroughly understanding the job. It might be appropriate in jobs where employees or salespeople handle valuable goods or have ready access to cash. In this case, business necessity might be argued that employees with prior credit problems are more likely to steal than people with clean records.

We’re done now, right? Nope! The EEOC and OFCCP like to see something called criterion validity. That is, they want more than your opinion. They want proof that scores accurately predict job performance. This can either be present-day (concurrent validity) or future-oriented (predictive validity). Present-day validation studies compare today’s credit scores to today’s job performance. Future-oriented studies require collecting (and ignoring) credit scores, waiting long enough to get a good reading on performance, and then comparing them.

Next, we have to define job performance. In the case of salespeople who handle cash we might use shortages. Or, we might examine shrinkage if salespeople work around negotiable goods. Employees who may be tempted to give concessions or award services might require deeper investigation. Every job has different performance criteria. The golden rule is if you cannot define it, then your reason for using it becomes weaker and weaker.

Scoring

Then we have the whole issue of how we use the scores we get. Should we use low, medium, or high bands? Maybe zero to one hundred? How about pass/fail? Do we look at only a few people or a whole gaggle? Banks for example, know sample-sizes can be misleading. They analyze huge numbers of borrowers and look for trends. Why do you think they ask you how long you lived at your last address or whether you owned a home? Taken alone, these tidbits provide little data, but when combined with other factors, they give the banks enough information to evaluate the risk of lending you money. (Contrary to what the media claim, the business of banking is lending money … they just want to get it back).

Just remember, every decision has its consequences.

Make Your Own Prediction

Here’s a prediction to think about. Which of the following organizations is more likely to be considered a socially bankrupt, adverse-impact, discrimination-monger? Company A that conducts traditional interviews and hires only applicants with good credit? Or, Company B that follows best practices by showing business necessity and job relatedness, uses validated tools, keeps documentation, tracks adverse impact, and makes ongoing efforts to reduce adverse impact?

Seems like a no-brainer to me.

from the A. C. Clarke foundation

I ended Part 1 with Arthur C. Clarke’s third law, “Any sufficiently advanced (hiring) technology is indistinguishable from magic.” It’s a play on words, but describes the same reaction I get from almost everyone unfamiliar with the best-practices outlined in 1978 in the Uniform Guidelines on Employee Selection Procedures.

Typical question, “How do I know this really works?”

Typical reply, “You tell me. If instead of working from a job description and doing a casual interview, I thoroughly define job skills, then use a variety of hard-to-fake accurate tools that evaluate whether a candidate has those specific skills, what do you think?”

Selling

After assessing hundreds of salespeople and sales managers (i.e., observing a full complement of their sales and management skills), I began to recognize some specific trends (I’m a slow learner).

Although they vary somewhat with type and position, they break down into just four general groups: 1) not being smart enough (or, conversely, being too smart); 2) being poorly organized and unable to focus (or conversely, being too nit-icky); 3) having insufficient people skills to manage, coach, sell, service, or work in a team; and, 4) having the wrong (or no) motivations to perform. (In some jobs physical skills are also important, but I’ll skip those for now).

Did you notice these factors have less to do with the environment or how they are treated than with the individual? And, for the “give ‘em a chance crowd,” they are very hard, if not impossible, to develop. That’s why these skills are so important to measure pre hire. If someone wants to take a chance on a salesperson who is dull, disorganized, has poor interpersonal skills, or is wrongly motivated, then be my guest … just do it with your own money, not mine.

How Do These Elements Play Out on the Job?

Here’s a reality check. First, have you ever felt “you know ‘em when you see ‘em”? If yes, then you are not alone. Now be honest, are the vast majority of the people who you hire successful salespeople? If your answer is, “ummm, no,” then I think we can all agree the “know-em-when you see ‘em” technique does not work very well. You were snookered by an empty suit (just like everyone else). Now let’s move on.

Not being smart enough is usually seen early (i.e., during training when the candidate has trouble understanding the product or service) or it can be seen late (i.e., when the candidate cannot develop an account penetration strategy, ask the right questions to discover a problem that needs a solution, or cannot keep up with changing technology). Here is an example: suppose on a scale of 1 to 10, where 5 is average, your product or service needs someone with a mental horsepower level of 7. Do you think a person with a 5 could do the job? How about someone with a 10? Not matching a candidate’s mental horsepower to the job is a sure path to disaster.

What about personal organization? Anyone who has ever worked with salespeople knows keeping records and following through on commitments is not one of their natural strengths. But at what level does it interfere with performance: a 3, 5, or a 7? It all depends on the complexity and nature of the job. Aim too high on the organizational scale and you get analysis-paralysis. Aim too low and you get customer service and delivery issues. I’m sure you have seen as many examples as I have.

Then there is the whole problem of interpersonal skills. Do you think they are soft and unimportant? If a big tree falls in the woods, does it crunch slow-moving critters living underneath? Being an effective salesperson requires exceptional interpersonal skills, starting with trust builders; and, they continue throughout the sales relationship as trust maintainers. Have you ever read studies that show customer loyalty tends to increase after a problem is successfully solved? Doesn’t it make sense to you that solving the problem and maintaining trust had something to do with that? Here is some free advice: never hire someone who claims they can sell anything to anyone, unless you enjoy dealing with buyer’s remorse.

Last of all, we have the motivational pieces. In my experience, effective salespeople are driven to compete and succeed. It’s as if they have a leaky self-validation bucket that always needs re-filling. Are candidates going to tell you they don’t have the right motivations? Sure, right after you invest time, training, coaching, materials, price concessions, and let them burn through dozens of clients and prospects. Of course, if you hire someone with too much motivation, you get Attila the Hun (or if it’s a female, Attila the Honey). If you hire too little, you get soggy milquetoast. This goes for providing customer service and a host of other motivational factors as well. What about hiring a highly motivated person with inadequate selling skills? Good luck with that train wreck!

Hopefully, these examples have convinced you “know ‘em when you see ‘em” decisions might feel warm and comfy because they are supported by the balance of consequences (remember, P-I-C almost always trumps O-U-D; see part 1 for an explanation of these acronyms).

Salespeople are great at selling themselves, but only occasionally deliver the goods. As you can imagine, measuring sales skills takes more than a casual interview, scrutinizing a W-2, or selling a wastebasket. It takes a combination of job analyses and hard-to-fake tests, motivations, and simulations.

Unless they just fell off the turnip truck, everyone who hires a new salesperson, puts an employee into a different job, or makes a promotion decision hopes for the best, but knows success will be hit or miss. Meanwhile, in the weeks and months between “hire the wunderkind” and “fire the jerk!” opportunities are lost; legal challenges increase; costly decisions are made; recruiting expenses continue; and, customers are lost.

It’s just another day at the office.

The benefits of best-practice hiring and promotions decisions are obvious. Screen-out a higher percentage of unqualified candidates, and survivors will be better salespeople. Better salespeople have lower turnover, need less training, and are more productive. Lower turnover, less training, and more productivity quickly lead to more profitability.

We Owe it to Employees

It may just be me, but I think someone who spends a substantial amount of time working for an organization deserves more than sink-or-swim treatment. That is, employers should to whatever degree possible ensure that employees and managers work in jobs which they enjoy and are qualified. Is it right, for example, for a salesperson to be bounced from one sales position to another because he or she doesn’t have enough skill to sell beer at an all-you-can-eat pretzel festival? How about a sales manager who cannot coach and develop the people who work for him or her? Has anyone ever suffered under a self-righteous sales executive with an acute case of narcissistic entitlement?

Someone in the organization hired these people. Someone was supposed to know what the job required. Someone had the responsibility to ensure every incoming candidate had the right skills. So, unless the charter of every organization is to hire unqualified people, we cannot put all the blame on the candidate. He or she just followed sorry advice to get the job and worry about performance later.

Like I said, it’s just another day at the office.

Hiring and Promotion Decisions: Balancing Consequences

Sticking to same-old-same-old hiring and promotion practices is wrong-headed, but can be explained. We are all born pre-wired to seek pleasure or avoid pain. It’s called the balance of consequences. I call  it the Ow!/Whee! syndrome. Experiencing an Ow!/Whee! moment depends on whether a decision affects you personally or other people (P,O), is immediate or delayed (I,D), and is certain or uncertain (C or U).

When hiring decisions are certain and immediate, we have a strong incentive to act. For example, a promising candidate applies for a position. We naturally think “Whee! … an open slot is filled and I‘ll feel good”: personal-immediate-certain (P-I-C). Sometimes Ow! and Whee! can occur at the same time. In this case, wrong decisions can also be made (Ow!). But “it will not affect me, is uncertain, and probably won’t be seen for months” (i.e., Ow!-O-U-D). What happens? Whee! (P-I-C) trumps Ow! (O-U-D) every time and we make too many wrong hiring decisions.

Foggy Understanding

Understanding how our psyche affects hiring and promotion is critical. Personal, immediate, and certain hiring and promotion consequences almost always carry the day. So let’s look closer at the sales hiring-chain:

Sales Managers’ immediate consequences:

• Usually hates recruiting because it takes time (Ow!-P-I-C)
• Thinks he or she “know’s ‘em when they see ‘em (Whee!-P-I-C)
• Thinks best-practice evaluation is burdensome (Ow!-P-I-C)
• Seldom calculates the personal toll of long-term sales failures (Ow!-P-D-U)

HR/recruiters’ immediate consequences:

• Searching for qualified candidates is hard and takes time (Ow!-P-I-C)
• It’s easier to measure results by quantity than quality (Whee!-P-I-C)
• Best-practice evaluation is burdensome (Ow!-P-I-C)
• Other people suffer with the consequences of long-term failures (Ow!-O-D-U)

In both cases, the Whee! and Ow! of hiring or promoting even the most marginally qualified candidate is almost universal. So, if we want to bring best sales hiring and promotion decisions to the forefront, we need to clarify long-term consequences: that is, show how consequences are actually personal, more immediate, and more certain.

Lifting the Fog

For a sales manager, tipping the balance between PIC and ODU requires a personal understanding that a stable of salespeople where 80% produce 80% of the sale is easier to manage and more productive than a stable where 80% produce 20%. Of course, I’m referring to the well-known 80/20 rule. The manager must come to grips with the fact that “know ‘em when you see ‘em” leaves a lot to chance because critical sales skills are assumed instead of verified. The challenge is getting him or her to sit down long enough to calculate the personal cost of not following best practices … and the personal rewards of following them, for example:

• Needs to remember the real payoff of following best practices is less coaching time, immediate productivity, and significantly improved chances of success (Whee!-P-I-C)
• Also, a bad hire takes more personal coaching time, has delayed (if any) productivity, and his or her probability of success is dim (OW!-P-I-C)
• Over the long run, almost all organizations that develop best-practice sales hiring systems will be more productive, and turnover will drop (Whee!-P-D-C)

I won’t belabor the HR argument, except to ask them that while line managers make the hiring decisions, who feeds them raw material? If the best HR can offer to line management is placing ads and half-baked pre-screening, then how hard is that function to outsource?  If litigation often goes hand-in-hand with poor hiring, promotion, and termination practices, what can HR do to reduce the risk? Yes, there is a Whee! or two for smart HR folks that will happen when:

• They recognize that other departments will find them more valuable when they do a better job pre-screening (Whee!-P-D-C).
• The legal credibility and the potential for litigation expenses drops (Whee!- O-D-C)
• There are people who generate more revenue (Whee!-P-D-C)

Best practices are not actually rocket science or mystical, but, as Arthur C. Clarke, English physicist and author, stated in his third law, “Any sufficiently advanced (hiring) technology is indistinguishable from magic.”

Think of best practice as accurate skills-definition and accurate evaluation.

That is a lot like hiring salespeople. You might think 80% of the salespeople would produce 80% of the sales, but that is seldom the case. In most organizations 80% of the salespeople generally produce about 20% of the sales. I know. I was a sales manager for a long time. I was also a sales trainer who used the best-known training and coaching programs I could find. Guess what? I never once saw a workshop or sales manager consistently turn poor salespeople into exceptional ones. It was 80/20 from the day of hiring to the day of firing.

Our organizations might have mastered machines and processes, but overall, they fall short in the human resources department. By that I mean we seldom, if ever, measure accurately candidates for the specific skills they need on the job. Instead, we take their word for it.

That’s why sales performance stays at a rock-solid 80/20 ratio.

Aseement: A Part of Evyday Life

People tend to think they are the best judge of other people. Normally, this keeps us out of trouble, but in jobs where specific human performance skills are required, it makes for disaster. Easy to fake pre-hire practices encourages hiring and promoting unskilled people and rejecting skilled ones.

Perhaps you noticed a few misspellings in the title of this section. Based on these errors, you might have jumped to a negative conclusion. Making sweeping assumptions based on insignificant snippets of information is human. If we like candidate X, we tend to assume he or she has positive skills (i.e., halo). If we see small mistakes, or dislike candidate Y, we do the opposite (i.e., horns). Both halo and horns decisions lead to dead-weight employees, abusive managers, and, narcissistic executives, to say the least.

We cannot avoid assessment … halo-horns decisions are pre-wired. Have you ever gone on a date? Both you and your date were in assessment mode. Have you ever taken a written driver’s exam, eye test, or drove around the block to get a driver’s license? You were being assessed. Have you ever taken a certification exam, completed an application, or answered interview questions? You were being assessed.

Forget for a moment the rumor that “assessment” is a virulent disease that hitchhiked to earth on moon rocks (e.g., I have it on the highest government authority that high-altitude weather balloons were involved); people are in assessment mode all the time. Assessment has been around since God made dirt. It happens every time a hiring manager or HR specialist asks an interview question, chooses a recruiting source, decides whether someone is worthy of promotion, reviews a job application, reads an ad, or checks a reference. Assessment done well is a blessing. Assessment done poorly leeds to organizational disaster.

Sales: The Life’s Blood of an Organization

Are you doing a good job avoiding sales candidate halo? Rank-order your current salespeople based on personal productivity. Next, subtract the right-time-right-place sales. Did you notice that a few folks produce a majority of the sales? Didn’t you expect everyone you hired to be a high producer … or at least perform equally?

Productivity differences are often the result of halo decisions. The low producers burn through potential clients and drain cash faster than a career politician. As an example, take a 10-person salesforce generating 5 million in sales. Four folks generate about 4 million and the rest generate about 1 million. Sales costs are budgeted at 10% of revenue, or about $500,000. The top four salespeople earn twice the commissions as their low-producing brothers and sisters. Assuming everything else is constant, we want to discover how much the bottom six sales people really cost.

Start by allocating sales costs by producer; that is, $500,000 /((4 people times 2x) + (6 people times 1x)) = 35.7K. Using this figure, we learn the top sales group costs $285,714 (4 x 35.7K x 2) and the bottom costs $214,285 (6 x 35.7K x 1). Next, we’ll calculate sales as a percentage of revenue. Top sales = 7 cents on the dollar ($285,714 expense/4mm revenue). Bottom sales = 21 cents ($214,285 expense/1mm revenue).

Our oversimplified math shows the bottom group costs three times the top group and generates 3 million less revenue. And, we have not even added the cost of recruiting, training, coaching, turnover, lost sales, poor customer service, and so forth.

Seven Steps to Hiring Successful Salespeople

Top salespeople have seven things in common: 1) the ability to quickly build and maintain trustworthy relationships; 2) skillfully asking questions to discover potential problems; 3) making persuasive recommendations and presentations; 4) helping people overcome purchasing risk; 5) having strong motivations to win; 6) willing to put the prospect or customer first; and, 7) doing all these things at the right time. Whew!

Look closely. Can any of these factors be ignored without sacrifice? Even if they could be accurately measured using traditional interview questions, do you expect a candidate to be honest? Suffice it to say unless each factor is thoroughly and accurately assessed pre-hire, we are doomed to be lifetime members of the 80/20 club. Breaking free requires hard-to-fake tests, behavioral interviewing, and simulations.

Decision Paralysis

So are you ready to start hiring top producers? A few readers may say, “Yes!”, but my experience predicts the majority will be paralyzed by indecision. After all, payroll money does not come out of HR’s pocket; low performers are someone else’s problem; horns and halo decisions are comfortable; people are afraid they will lose control; filling slots gets more emphasis than hiring qualified people; and, in spite of the rank-ordered facts to the contrary, many sales managers insist they “know ‘em when they see ‘em!” Meanwhile good candidates are turned away and bad ones are hired.

Is gut important? Of course! The gut is always the final form of assessment … but, right or wrong, guts are always 100% convinced of their infallibility. Hiring professionals cannot afford the luxury of uninformed guts. There are good reasons why every organized sports team only hires players who pass tryouts. No skills, no play. The same goes for many other professions where the costs of unskilled people are too big to ignore.

So, unless your organizational objective is to pay for attendance instead of performance, your job is to thoroughly and accurately assess every sales candidate for each of the seven factors. Yes, you will kiss more frogs to find your prince or princess because only one about 1 in 10 sales candidates can perform all seven satisfactorily; however, the newly hired royalty will produce like champions.

One last note: depending on their involvement in the setup process, expect a few hiring managers to override some of your early rejections (e.g., it’s not easy to control uninformed guts). But, fast learners don’t make the same mistake twice.

Discover

The first step in test development is to know which KSA to measure. Is it physical coordination or abilities, staying organized, learning new information, making new decisions, having the right attitudes, getting along with people, or something entirely unexpected? Often, people mistake results for skills. Overall, results are what we expect eventually, but ignoring the KSAs that made them happen is like enjoying a box of fresh Krispy Kreme doughnuts. Those artery-clogging pieces of heaven don’t make themselves. It takes time, good ingredients, the right equipment, a good recipe, and skilled doughnut makers (e.g. the KSAs of delicious doughnuts). Likewise, employees don’t bring “results” to work. They bring KSAs.

Sometimes, if they don’t develop a test themselves, managers might trust a test vendor’s marketing claims. Buying a generic test for a customer service rep, manager, or truck driver might sound simple, but can be as problematic as developing a test on your own. We can all cite examples of jobs with the same title that are very different, as well as jobs that are similar but with different titles. In the worst case, many tests either measure the wrong thing or cannot predict job performance (there is a good reason why the DOL “Guidelines” state that users should not accept vendor claims of test validity).

Predicting job performance requires measuring critical KSAs. If the test (or behavioral interview) shows the candidate cannot solve problems, then the hiring manager can reasonably assume the candidate could have trouble solving on-the-job problems. The same goes for every other KSA. In short, only measure what you need — and measure it accurately. If not, skip it. Let’s never forget the whole purpose of pre-hire screening is to learn enough about a candidate’s skills (not results) to predict performance. Unfortunately, some things simply cannot be measured: unexpected problems at home, health issues, crazed managers, unexpected market forces, and so forth. Those will always be wild cards.

Develop

It’s generally a good idea to avoid open-ended interview questions (the candidate will have had plenty of time beforehand to rehearse good answers). It’s also a good idea to avoid self-reported questions unless you can prove the candidate is either not lying or living in an alternate universe. If you are a skilled behavioral or situational interviewer, you might craft questions seeking examples of when, where, and how the candidate applied KSAs you need for the job. It’s not the most accurate tool, but highly structured interview questions usually provide better data than unstructured ones.

Sometimes home-grown tests take the form of case studies or tests of problem solving, planning, numerical ability, word-association, and so forth. Written tests are not as flexible as structured interviews. On the other hand, they are very hard to fake when professional development standards are followed.

Professional standards require questions that do not overlap; stable scores over time; items that are neither too easy nor too hard; questions of varied difficulty; individual items strongly correlated with overall scores; standardized answer keys; and, proof the test measures what it is supposed to measure. In technical terms this is referred to as test reliability, inter-item reliability, and validity. In lay terms, it means you can trust the scores to be consistent and predict specific KSAs.

For those managers who don’t know what they don’t know about test construction, there is a whole book on it. Ignore its principles and your test will probably be either one big waste of time or a lightning-rod for an expensive legal challenge … and rightfully so. If a manager insists on developing his or her own test, tell him or her to be sure to follow the “Standards“ and the “Guidelines” (do a search on the 1978 Uniform Guidelines on Employment Selection Procedures). If he or she thinks these documents are too hard to follow, suggest they document their reasons for ignoring them. Both executive management and the legal department will probably want to know why the organization is using worthless tests and unnecessarily exposing itself to legal challenge.

So what kind of testing works best? It depends on what you want to measure. In my experience it’s a good idea to start with a few highly structured interview questions (if you are skilled in behavioral interviewing) and show a realistic job preview. If you have a large applicant pool, you might even use a smart web-screening app. Follow next with hard-to-fake tests such as attitudes, interests, and motivations, critical abilities, and structured simulations. Save time by arranging your tests so people can drop out if they fail any step. Testing is not limited to written format. Testing also includes interviews. Be specific. One-size-fits-all seldom fits anyone.

Follow Up

Two common mistakes include either not following up or assuming “higher scores are better.” Mistake! This is like buying any-size clothing expecting they will fit. Setting scores is dicey. If they are set too low, candidates will probably be under-skilled and troublesome. If they are set too high, candidates will probably be over-skilled and troublesome.

If the test measures mental ability, it is likely that some demographic groups will fail the test at a higher rate than others (e.g., brace for adverse impact!). If the test measures something that could be learned in a short period of time, you might be asked to show why potentially acceptable candidates were excluded. Sometimes score-setting is as challenging as test development.

For example, one of my clients decided to set scores lower than necessary just to get more protected-group applicants into their training program; and, another, after using a home-grown spelling and grammar test (e.g., with an overall passing rate in the high 90’s) wondered why new hires could not “spelle good.” If tests (and interview questions) are supposed to predict performance, verify they work.

Recent OFCCP and EEOC audits show the government is becoming more sensitive to criterion validity issues. That is, employers have been increasingly required to show legitimate links between test scores and on the job performance. Not to mention the fact the current administration has increased both the size and budget for adverse impact investigation and enforcement — much of which seems to be politically based.

Hiring the best people is a lot of work, but in the end you will have stronger legal credibility, lower turnover, higher individual productivity, and less training. In addition, your workforce will not only be diverse; every member will be highly qualified. Home-grown testing is like buying expired food. It might seem like a quick way to save a buck, but it always proves to be a big mistake.

About Metrics

The only reason to collect metrics is to make informed decisions: am I doing well? What do I need to improve? Are my activities helping achieve organizational goals? And, so forth. Metrics are not the “cool” thing to do, nor are they an interesting thing to play with. Metrics should provide specific feedback in specific areas that are actionable.

Consider quality initiatives. Many manufacturers measure product quality at the end of the production line. This is problematic because there are many small steps along the production line that could affect the final product. Measuring quality at the end of the line tells you whether the product is good or bad, but almost nothing about how to fix it. About 50 years ago, W.E. Deming and J.J Juran became leaders in quality improvement by breaking manufacturing into small process steps; collecting data at each step; working to control the process variability; and repeating the process until it was under control. In my experience, HR needs to incorporate some of their ideas.

To incorporate quality control into hiring, we must identify specific job competencies (i.e., the “how,” not the “what”), evaluate the accuracy of each testing-tool, evaluate demographic impact, gather performance evidence, work to control the variability, and so forth. Evaluating hiring quality takes professional practice. So why is it not done more often? I can only think of four reasons:

1. People don’t have a clue how to do it
2. They think it’s too much work
3. They only care about surviving the guarantee period (this is what I hear most from external recruiters)
4. They did not think of it themselves (see #1, #2 and #3)

The Status Quo

It does not help that people think recruiting is a field one can “break into” more than one can break into law, medicine, science, or engineering. So old-school recruiting practices remain strongly rooted in reviewing a job description, interviewing a hiring manager, and using HiHowAreYa interview questions. The usual result is unclear job skills that must be inferred from resume data or taken at face value. In the final run, candidates are often compared with one another instead of to the job, and usually the ones who walk and talk most like current employees get hired. As a result, only about half of new hires fail to achieve high performance. A few months or weeks later, enterprising internal recruiters send out one-size fits-all smile sheets to hiring managers expecting to get honest feedback from people who are highly motivated not to admit any kind of bad hiring decision. Is it any wonder why status quo metrics seldom tell you very much?

Useful Metrics

Let’s start with identifying specific skills that lead to results. This is done when a job analyst interviews job holders and their managers about what it takes to do the job; what leads to successes; and, what leads to failures. This is a learned art because the analyst must know how to listen for clues associated with competencies that either precede or are associated with performance. For example, a long story about a successful project may involve learning, decision making, interpersonal skills, planning, and so forth. It’s up to the analyst to listen to each story to isolate the key competencies against which new candidates should be measured.

Specific competencies serve as the least common denominator against which all metrics are collected. For example, metrics will tell us if our job analysis missed anything; if the job changed; if our tests and measures (i.e., that includes interviews) were accurate and trustworthy; what needs to be modified; if pass/fail scores were set too-high or too-low; or, if any of the tests and measures adversely impacted protected groups. And, yes, this includes manager-employee fit.

Foggy Bottom

Military experts often refer to the fog of war. That is when everything is happening at once and nothing is clear. We can use the same term for the fog of performance: foggy because managers’ ratings are often part opinion and part fact; hard performance numbers often conflict (e.g., customer satisfaction and time per call are often negatively related); and, results-data often occur long after competencies are applied. Nevertheless, a skilled analyst knows how to gather highly useful and actionable data that can be used to improve and modify quality of hire.

If you don’t know specifically what to measure, how to measure it, and what to do with the results, then metrics can become a muddy mess. It takes clear job analysis, validated tests, structured interviews, and effective manager feedback to make hiring metrics work.

Sure, we should plan ahead, forecast hiring trends, and develop candidate pools. This is just good business sense. But, assuming hiring managers and staffing folks are doing the best they can, that is not enough. If we do not abandon old ways of thinking and adopt new tools, articles like this will continue into the future. Let me begin by restating a few obvious facts:

• Interviews range from highly structured to ROTFL. Although they get better with structure, interviews are still tests.
• Job requirements are almost always taken from an old position description. Questions are delivered orally and answers are delivered orally. Scores are almost always based on personal opinion.
• In the end, interviewers usually compare candidates with each other instead of to the job.
• No one intentionally hires someone who cannot do what was expected.
• People promoted based on performance as individual contributors seldom become good managers.
• A poor hiring program leads to a shallow promotion pool.

If sports franchises used organizational hiring practices to hire players, they would hire golfers, send them to soccer workshops, manage them as if they were fly fishermen, and reward them based on their ability to play badminton.

We can cite more facts, but the obvious question is not whether or not we should do a better job hiring and measuring candidates, but how can we start doing it?

Think Outside the Box

There is an old training problem where people are asked to connect 16 dots arranged 4X4. They are told to use only four straight lines and to keep the pencil on the paper at all times. The task is impossible without going outside the box; so is fixing the low-performer problem. Going outside the box with employment means: 1) doing a better job defining how a job is to be done; 2) using tests that measure “hows”; and, 3) following up on specifics. And, guess what? The good part is that’s what the Feds want you to do anyway!

Clear the Competency Cobwebs

Start by tossing-out junk competency definitions. Unless the old job is exactly like the new one, the only competencies that reliably can be used to bridge skills from one job to the next are technical knowledge, cognitive abilities, planning skills, behaviors, and motivations. I’ll leave physical skills for another article. In short, you need to know if the employee is smart enough to solve problems in the new job, knows the right things, can effectively plan and organize work, has the right interpersonal skills, and wants to do what’s required. It’s really so basic that some people have trouble understanding it.

You see, asking about results is the part that gets our attention. Asking “what have you done” is much easier than asking “how did you do it?” And, asking “how did you do it?” is easier than knowing if the candidate is telling you the truth. And so it goes. Even while our human nature keeps telling us how important it is to get to know the candidate, our job responsibility is to get to know whether the person has the skills to do the job. I know I don’t have to cite examples. We all have an abundance of them.

Start by understanding how a job should be accomplished. That is the secret of what you are looking for. And, while you are at it, ask a few people who actually do the job. You would be surprised at what you can learn.

Master Your Tools

Interviews are quick, flexible, cheap … and inaccurate. If they are the only tools you use, abandon any idea that you can improve hiring quality. It will save a lot of frustration. Of course, adding structure to your questions without adding job analyses data or standardized scoring may make you sound more professional, but not knowing what to probe for or how to evaluate it will still fall short of your goals. Get to know as much as you can about hard-to-fake tests such as simulations and ability-type tests. If you don’t know how to use them, learn.

Avoid the Junk

I really feel sorry for someone who seriously attempts to navigate today’s test market. It is filled with so much junk and misleading information that it is almost impossible to make a well-informed decision. In general, avoid any test that claims it is approved by the EEOC, does not have adverse impact, states it can “help” you make the right hiring decision, has special occupational norms, and so forth. These are red flags. Even if you don’t get sued for hiring discrimination, you won’t be so lucky defending a wrongful termination charge or an internal discrimination challenge. Of course, there are all those bad-employee expenses that John Sullivan cited in his article.

Playing the Odds

Every hire is a gamble, and no system is perfect. Your only choice includes whether to continue using non-predictive interview techniques or learn better processes that screen out a greater percentage of unskilled employees. This is called validation. Nothing in the organization’s arsenal delivers the same ROI as a good hiring system. Just imagine instead of having a typical organization staffed with 20% top producers, 20% bottom, and the rest in the middle, what it would be like having 70-80% top producers.

Of course, you could always continue complaining about the high cost of low performance.

Litigation vs. ROI

Litigation threat has down-the-road implications for developing sound hiring and promotion processes. Attorneys seldom work for free, and you do not have to lose in court to lose money.

I have never personally met an attorney who was experienced in job analysis, conducting validation studies, or documenting assessment processes. I’m sure there are a few, but it’s generally not their specialty. Attorneys are trained to know the law and to argue persuasively, not design, develop, and validate selection systems. The corporate attorneys I have worked with fully appreciate the benefit of a well-documented job analysis, validation study, and assessment process. To quote one of them, “I would much rather defend <this process> than the one we use now.”

Remember, in the litigation world, making a persuasive argument is more important than making a good employment decision. However, in the organizational world, if you don’t make a good employment decision, you get to pay twice: once for litigation and forever for low performance.

No Better Than Chance

We said earlier that people tend to think interviews and tests are two different things. Fortunately, for them at least, interviews fly under the validation radar because most folks think of them as conversations. Unfortunately, once an unstructured interview has screened out the blatantly unqualified, it has a long history of chance-level hiring decisions. Think of it this way: after the candidate has passed on unstructured interview, you might just as well ask him or her to pick a marble from a jar filled with 50 red marbles and 50 blue. Blues are high performers. Reds are low. Unstructured interviews are a significant blind-spot. We all know that.

Validation Designs

You don’t need to know the details, but you do need to know there are different kinds of validation. They include predictive designs where everyone takes a test, the scores are ignored, and job performance is later compared with test scores; concurrent designs where job-holder scores are compared with job performance; individual comparisons; group comparisons; group averages and score distributions; measuring job content; evaluating mental constructs; assessing OTJ performance criterion; examining the face of the test; and, so forth.

I prefer predictive designs using OTJ performance, but organizations seldom have the patience to wait; designs using current employees suffer from technical problems (see range, below); and, defining performance is always an issue no matter what design is used.

Just remember that there is no single type of validation. It varies with application; the number of people involved; the potential exposure for litigation; the importance of the job; and, so forth. One last point: Taking a broad-scope generic test, giving it to all job holders, and developing a high-group norm and a low group norm is not validation. Why? Validation requires a causal relationship. Like hemlines and the stock market index, other than moving together, if one does not cause the other, numbers do not make it valid.

Performance Considerations

Next, there is the problem of “what to measure”; that is, the data used to validate, or verify, the test actually works. This includes hard data like turnover, individual production, new account generation, business expansion, call time, customer satisfaction, and so forth. Hard data is always nice to have, but we have to remember it often conflicts, is part subjective, and part objective. Data taken from performance reviews are usually frustrating (i.e., useless). Everyone tends to look the same on paper.

We may have to make compromises and adjustments along the way, but, if test scores cannot be compared with performance, there is no way to validate the test. You might as well stop testing, buy a jar of red and blue marbles, and save your money. Your employees will be embarrassingly average, but I’m sure there is a sharp litigator somewhere who might be able to make an effective argument for using marbles.

Range Restrictions

Validation is always confounded by the problem of restricted range. Restricted range means differences between high and low performance among job among job incumbents will always be much less than among job applicants (e.g., think of skill differences between pro golfers and skill differences between people in the gallery). Ideally, we want to compare a broad range of test scores with a broad range of performance ratings.

Restricted range plays havoc with analysis because, instead of having the luxury of big differences between best and worse, the analyst must examine teensy-weensy test score differences and teensy-weensy on-the-job-performance differences.

Group-Size Prerequisites

I won’t go into statistics except to say that trust drops fast if the differences are small and there are insufficient people to evaluate (i.e., I prefer 25 people per factor); if you try to measure too many different things with the same test; if the test domain does not actually lead to or affect performance; if both performance and test scores are not normally distributed; or, if the groups are unequal, then you cannot trust your analysis. For example, comparing scores of five Pandas, 25 Penguins, 12 Puppies, 18 Kittens, and three Bunnies might give you impressive looking numbers, but they will be junk.

Assessment Is not a Four-Letter Word

I listened to an excellent webinar last week. It addressed accurate hiring and placement techniques. The presenters cited substantial payoff in ROI measured by turnover, productivity, training, performance, sales, and so forth. I’m sure many C-level executives would give up their Mercedes for a week in exchange for the financial benefits presented, but there were less than 100 people in attendance! And half those indicated they were already using assessments. What’s up with that?

I think the HR community considers assessments in the same class as toxic waste: dangerous, threatening, difficult to handle, and expensive. Well, let’s put that to rest. Assessment is just another word for “measurement,” and, measurement takes place every time a candidate is screened for a job. Don’t forget an unstructured interview is an assessment (although a rather poor one).

The webinar folks argued the same point I have been trying to make for years: accurate (i.e., validated) tests and assessments lead to better hires, and that leads to reducing expenses and increasing revenue. Is it expensive? Compared to what? How would you feel about an ROI of at least 100% within the first one or two hires? And, non-stop payback after that?

Organizational Handicapping

Here are some of the symptoms of using low-accuracy assessments, failing to validate the assessments you are now using, or selecting employees based on demographics instead of individual skills:

• Not enough people with the right KSAs for promotion (i.e., increasingly complex jobs require increasingly complex KSAs)
• Increased potential for litigation and adverse impact (i.e., decisions are based on personal opinions are less predictive than legally credible validated data)
• Frequent training requests to fix “broken” (i.e., unskilled) employees
• Weak bench strength limits organizational flexibility and reaction time.

The Final Product

I want to leave you with these thoughts. If test or interview questions are not validated, then it is not possible to know whether they work or not. There is no such thing as an EEOC or OFCCP pre-approved test. There is no such thing as a generic test validated for your industry (unless, in the unlikely event, you can show the two jobs require essentially the same KSAs). And, managers’ home-brewed tests are something to avoid like the plague. Finally, investing in validated tests, interviews, and assessments can yield the single best ROI of any organizational dollar you could imagine.

Can you imagine what it would be like if management considered your department a revenue generator instead of expensive overhead?

Tell Me About Yourself

Asking someone to “Tell me about yourself” does not sound like a test question. But, what would you call asking a question, evaluating the answer, and making a decision? It makes no difference if it’s written on paper or verbal. If you make a decision based on a candidate’s answer, it’s a test. Now, how about this kind of test question:

“Give me an example of when you solved a difficult problem. Tell me what the problem was, what you did, and the result.”

Better, yes? But only if the interviewer knows that problem solving is important to the job; the kind of problem solving required; the difference between good and bad problem solving; can pry details from the candidate; and, uses a standardized scoring sheet. Why get all nitpicky? Structure is the best way an interview question can be a good predictor of job performance.

You may think unstructured interviews are the bread-and-butter sandwich of recruiting, but look between the slices: usually you’ll find mold and rancid butter. Interview questions need validation just as much as formal tests.

You cannot trust any selection tool that is not valid! Consider Merriam-Webster’s definition of valid: having authority, relevant and meaningful, appropriate to the objective, supported by truth, basis for flawless reasoning, evidence, justifiable. Now, isn’t that something you want?

False Sense of Security

Once upon a time I worked for a large consulting company. It was filled with administrative assistants who could not spell and used bad grammar. After listening to client complaints, I went to the internal HR department and asked if they gave AA applicants a spelling and grammar test. “Yes,” they answered, “We use one developed by the owner.” A quick examination showed the test looked OK (i.e., was face valid) so I asked if I could see the scores from the last 100 hires. Guess what? Passing rates averaged about 95%!

High AA scores might have given HR a warm and fuzzy feeling, but it was just another case of organizational incontinence. I considered giving them a box of departmental-sized Depends, but I think they would have missed the point. Their test did not test anything! It was not valid. While worker bees labored to present a professional image to big-buck clients, incompetent AAs were misspelling words and using atrocious grammar. I’m sure the president would have responded if he had known, but HR was not going to rock the boat if their life depended on it.

Self-developed tests may seem like a good idea, but they are usually inaccurate, invalid, or poorly maintained. Bogus tests harm the organization because they give a false sense of security, while actually doing nothing to improve quality of hire. This is especially true when managers get frustrated and decide unilaterally to make up their own test. If it’s important enough to test, it is important enough to validate. Otherwise, forget it. Even high-profile assessment organizations make foolish decisions.

From the Frying Pan Into the Fire

Math and reading are becoming problematic. I’ve heard from dozens of organizations about employees who cannot read, calculate, or write. This is an issue when becoming automated, adopting computer-driven equipment, or encountering frequent or steep learning curves. In response, unknowing people think grabbing a test off the shelf will solve their problems. I’ve even seen some who used reading tests developed for placing students in the right English class.

Testing studies show a three-bears effect. That is, human KSAs come in sizes: too little, too much, and just right for the job. For example, we know intelligent people tend to perform better than unintelligent ones; and, intelligent people tend to score higher on abstract verbal and numerical tests. But now life gets challenging …

It’s a fact of life (at the group level) that intelligence test scores cluster into different curves depending on demographics. There are plenty of theories why, but we’ll conveniently ignore them. Let’s just say we have five demographic groups: Pandas, Penguins, Puppies, Kittens, and Bunnies. Pandas score an average of 85, with 2/3 falling between 70 and 100; Penguins average 93 with 2/3 between 77 and 107; Puppies average 100 with 2/3 between 85 to 115; Kittens average 107 with 2/3 between 100 to 122; and Bunnies average 114 with 2/3 between 107 to 129.

Demographics membership does not force someone to be smart or dull. Individual Pandas can still score substantially higher than individual Bunnies and individual Bunnies can score lower than an individual Penguin. There will just not be as many high-scoring Pandas and Penguins at the group level than Kittens and Bunnies. Now this next part is important!

We don’t need to be rocket scientists to know that low scores lead to mistakes, bad products, and safety violations, while high scores usually lead to boredom and turnover. Balancing demographic differences with our need for “just-right” intelligence, how do we establish and defend cutoff scores?

No organization I know is forced to hire unqualified people. But, the EEOC and OFCCP expect you to show there is a business need and job requirement. Oh yes, and it is incumbent on employers to give new employees “reasonable” time to learn the necessary skills. If you eliminate new employees based on something they could learn in a reasonable time, you better be able to explain why.

A well-done validation study keeps the input funnel filled; employs only fully qualified employees; keeps training times reasonable; minimizes adverse impact at the group level; and maintains both business necessity and job requirements.

In the next part, I’ll discuss a few differences between validation and litigation.

Developing skill sets is the best long-term strategy — like allowing time for traffic backups and learning alternate routes.

Welcome to the Audit

A few days ago, you might have listened to a webinar presented by an attorney who specializes in OFCCP audits; over 400 people did. He made some important points. For example, the Obama administration is taking a broader view toward adverse impact; computerized systems make it easier to extract applicant data (which unfortunately can be used against you); there is more emphasis on writing better disposition codes; you might soon find yourself defending against white males claiming discrimination; there is increasing government emphasis on hiring veterans and people who are disabled; and, last (but not insignificant), OFCCP audits generate substantial revenue.

It Takes More Than Creative Coding

Enlightening and timely, but while most of the attention was on better coding, I think a wise recruiter might have seen that disposition codes, like scores at the end of the game, leave a lot of questions unanswered. For example, the speaker advised recruiters to include in their code at every step the reason why a candidate was disqualified, and who did it. Sounds like a new idea, right? Well, check out this link. You will find some of the information eerily familiar to the conference call. By the way, you might want to start preparing your argument to management about why you have ignored something that has been around for about 32 years. But, let’s put on your OFCCP investigator hat. Wouldn’t you want to know: why include this step? What kind of training did the decision-maker have? What kind of scoring system was used? Was it standardized? Where is the professionally conducted study defining job requirements and business necessity? Where is the validation study showing this test/interview/exercise predicts on-the-job performance? And so forth. You might be thinking, “This is nonsense. It’s too much work!”

My response would be, “If your job is finding the most qualified candidate, with what part of this do you disagree?” Anyone who has ever worked in an organization knows first-hand that human performance varies from low to high. I won’t bore you again with the financial cost of low performance, but doesn’t it tell you something when low performers manage to get through the same screen as high performers?

Demographics and Job Qualifications

At least (as of today) organizations are not under any mandate, government or otherwise, to hire people who cannot do a job. Let’s repeat that. You are not required to hire unqualified employees. Once again: organizations are not required to hire people just because they are old, young, male, female, Asian, white, black, or any other demographic, if they are unqualified.

By now, you might note emphasis on the word “qualified.” That is where organizations of all shapes and sizes get into trouble. They fail to clearly define “qualified.” Establishing job qualifications and trustworthy measurement tools are clearly spelled out in the 1978 Uniform Guidelines on Employee Selection Procedures: the gold standard for defining job requirements, establishing business necessity, choosing tests, and writing interview questions. And don’t forget the 1999 APA Standards for testing. It’s the gold standard for designing and administering tests. Never heard of them? Don’t think they apply to you? You may not know it, but if you have anything to do with recruiting, they define how to do your job!

Know Your Essential Elements

Next, a note about people with disabilities. Hiring the disabled is covered by the 1990 Americans with Disabilities Act, better known as the ADA. Basically, the ADA says employers should define essential physical elements of the job and be ready and willing to provide reasonable accommodations to job-qualified disabled people. It goes something like this: you define job requirements, you evaluate/assess the candidate for the job, and you ask them if they can perform its essential physical elements with or without reasonable accommodations. There is more, but you get the drift. Give disabled applicants a fair shake.

How do you establish essential elements? You have to do an ADA-type job analysis that focuses on things like vision, movement, strength, agility, and so forth.

Doing the Right Thing

Following all those pesky rules is not only the right thing to do for an audit, it’s the right thing to do for the organization, the candidate, and the profession. Auditors make sure your organization treats candidates (i.e., voters) fairly. Since, at a demographic level, not all groups perform alike on tests/assessments/interviews, it is easy for the auditors to find statistically significant evidence of adverse impact based on gender, race, age, and so forth. Imagine how your organization would function if it ignored skills and hired only on demographics. Where would your future leaders come from? How much money would you pour into training the un-trainable? How many extra people would you need to do one job? What about your competitive ability? Can you spell b-a-n-k-r-u-p-t-c-y?

Get a Grip on Hiring

There is more to hiring the right people than casting a wide net and getting to know the candidate. Read the Guidelines. Read the Standards. Read the ADA.

When you are finished, it’s OK to go outside and throw up. You won’t be alone. As I said before, you may not like what you read and you may not want to do that much work; but, you have to ask yourself what part you disagree with: documenting job requirements and business necessity; identifying essential job elements: training decision-makers; believing false vendor claims; standardizing scoring methods; or validating your interviews/tests/assessments? When the day comes you have to defend poor hiring practices, it will cost your organization big bucks. You can always hope they blame it on someone else.

In the meanwhile, you can always decide to conduct business as usual, staffing your organization with more people than you need to do the same amount of work, hiring the bottom 80% of salespeople, promoting fully unqualified people into management positions, enjoying higher turnover, spending more training dollars, making more mistakes, crippling competitive responsiveness, building a shallow succession pool, turning away fully qualified disabled candidates, and continuing to argue with line managers about wasting their time with unqualified candidates.