“The successful applicant will have 10 years experience in the widget industry.”

Hogwash!

The end result of this approach is that companies hire the industry retreads.

Perhaps, employers think that this person will bring along valuable competitive secrets — maybe even some clients. While that may occasionally happen, this approach negatively impacts the company. They may as well hang a sign outside that says, “No new ideas permitted” because that is what you get when you focus your search on industry people only. What often happens is that the individual gets hired because they can create the illusion of brilliance by using industry jargon to blind the interviewer. “Eureka! We’ve found our sales manager! She is very strategic!”

Every company thinks they are in an industry that is so unique and has so many nuances that the hire must have industry background. But most industry information can be taught. The company needs to get over its hubris thinking that its industry is so special that it takes an industry veteran to be successful.

Product knowledge is not the main driver in a successful salesperson, nor is it the primary one for the successful sales manager. CEOs bounce from Fortune 1000 company to Fortune 1000 company based on their CEO acumen, not their industry knowledge.

A more prudent approach for hiring the right sales manager is to look for a candidate who comes to the table with the specialized skill-set associated with a sales manager. This is a specialized skill set that is often portable to any industry. The role of the sales manager is to both be a leader and a manager, which are not usually skills developed in the womb; they are cultivated and developed through training and experience as a sales manager. Some of the elements that companies should be focused on when hiring the right sales manager include:

• Recruitment. Whether the company has an opening on the sales team or not, the best sales managers are on a never-ending quest for strong talent. As the prospective employer, you want to understand the candidate’s process for screening sales candidates. How do they prime the applicant pump? Can they develop a profile of the ideal salesperson, and prioritize it between required and desired attributes? What is their process for evaluating candidates against the profile? Ask any company why they miss their revenue targets and most will tell you that having unfilled slots on the team is a contributing factor. Recruitment is a very important arrow in the sales manager’s quiver.

• Onboarding. Rarely can you hire a salesperson, hand them their territory, and send them off with a good luck kiss. Not if you expect them to be successful. Another key skill of the sales manager is their method for quickly assimilating the salesperson into the organization. What is their strategy to minimize the amount of time that the new salesperson is in a non-revenue generating capacity? What is their plan to make them productive in the least amount of time? How do they measure whether or not the neophyte salesperson is going to be successful?

• Process. Many companies have one superstar on their sales team — their rainmaker. That’s not exactly a scalable model. It limits growth and creates exposure for the company if the rainmaker leaves. Scalable sales organizations are based on process. The entire team follows a specified model based on a defined formula. Find out if the candidate can create this process for the company, what experience they have in doing so, and what the results were.

• Metrics. The wonderful aspect of sales is that there is so much data that can be reviewed to understand trends and make changes to the business. While interviewing, scrutinize how the sales manager uses metrics in their approach. See how they have used metrics to affect performance of their team. Learn their approach to scrutinizing a sales pipeline or forecast.

• Compensation. The beauty of sales is that the compensation plan serves as the salesperson’s job description. This can also be a curse for the company if the wrong behaviors are rewarded by the plan. This is another important skill that a strong sales manager should possess. Find out their approach for developing the right compensation plan for the company. See how they determine which behaviors to reward, when, and how.

• Skill development. Sales is philosophy, so no one ever knows everything about it. It’s also very easy for salespeople to develop bad habits. Thus, the sales manager should have a skill development plan for their team. Get to know their approach for developing their team members. Probe how they inspire the overachievers to continue to overachieve. Ask they manage the underperformers and lead them to either perform or deselect from the company.

• Leadership. The first six items fall into a management category. However, the strong sales managers are also leaders. Their sales teams will run through walls for them. Their salespeople not only want to be successful for themselves, but also for their manager. Determine how this sales-management candidate creates an environment where others are inspired to follow them and their teachings. Leadership skills and sales force retention work hand-in-hand. Strong leaders keep their strong players on the team for the long haul.

In addition to cultural fit, these are the seven key elements that a company should use to make a decision to hire a particular sales-management candidate. What the employer will get with this hiring approach is a strong, scalable organization with fresh ideas.

But many folks predict that it’ll be increasingly harder to find qualified applicants to fill job openings, especially when it comes to white collar, managerial, and professional-level jobs. We have all dealt with numerous situations where the age-old “mirror test” (hiring anyone who is breathing and thus has the ability to fog a mirror) is the only thing required to fill a position. In such cases, it’s harder to make an argument for using something that will actually serve to further reduce the number of applicants to be considered.

Assessment can have value even when one has few applicants to chose from or even when there is a shortage of qualified applicants. Below are five good reasons why it still makes sense to use assessment, no matter what the labor market looks like.

Reason 1: Assessment can be part of a good sourcing and branding strategy

Assessment has begun to occupy new places within the hiring process. Over the past five years we have seen it begin to be folded into the job searching and matching process. Assessment is a great way to help match job seekers to openings that are a good fit for their background, skills, and values. Even in a tight labor market, job seekers can still benefit from some direction to help them make important decisions about what jobs and organizations are right for them.

Compelling employment branding is the first part of the equation here. Much employment branding seems to be a regurgitation of the same old song and dance. What company doesn’t value diversity? What company doesn’t care about the environment? Assessment can help us to get past the fluffy, generic employment branding stuff and get right to the heart of the matter: matching people with the right employment situation.

Recently, assessment has become a key ingredient in “values matching,” a branding and sourcing strategy that can really make a difference. Values matching will continue to be a hot area because job applicants wants to ensure they “fit” with the culture and value of the organization. Assessment is an excellent way to provide a index of “fit” that can be used to help applicants to understand how well what they may fit with an organization, or even with a specific role, job, or workgroup within that organization.

Organizations that fill their hiring funnel with applicants who possess congruent values and have some degree of fit will find it easier to be sure they hire persons who have a good chance of sticking around longer and being more productive. Values matching can be used as part of an initial sourcing strategy by including it as a key element in the creation of job profiles used to match applicants with openings or as part of the search process on corporate web sites. Even if there is only one applicant for a position, if that applicant is a good fit, hiring them represents a good decision for all parties involved.

Reason 2: Assessment can still provide insight needed to support good decision-making

Suppose there are only two applicants for one open position. Whomever is making the hire still must make a decision between the two applicants. The decision-maker can still benefit from having some data to better understand each of the applicants from which they have to choose. Most employment decisions are made using resumes and unstructured interviews. These are the two most common tools available to hiring personnel. While both of these tools do have some value, they function best when used in conjunction with information that can help those doing the hiring to read between the lines. Assessment is a perfect tool to help those making hiring decisions to better understand each candidate. Informed decision-makers are more likely to make accurate decisions than those who must make decisions in the absence of good data.

The results of even a basic assessment can provide decision-makers with additional data points to help them make a more accurate decision. The number of applicants one has to choose from does not alter the idea that informed decision-making is the way to go. So, even in a tight labor market where one is lucky to have even a few applicants, it’s still valuable to provide decision-makers with tools to help them better interpret subjective information such as resumes and unstructured interviews.

Reason 3: Assessment can help you hire for potential

One common strategy in tight labor markets is to shift focus from hiring for a specific set of skills, abilities, knowledges, etc. needed to do a specific job toward hiring for potential. In such cases a few key abilities or characteristics are identified and applicants are evaluated based on their ability to bring this valued “raw material” to the table.

This change in focus allows the organization to hire those who may not have had the exact experience needed to do the job for which they are applying, but who have what it takes to learn the job. Such a strategy requires a strong focus on training and development. Still, such a program allows organizations to take a broader focus when looking to fill open positions. College recruitment and hiring programs are an excellent example of this. Hiring individuals straight out of college allows companies to fish in a bigger pond. Assessment is an excellent way to assess potential. Basic cognitive ability and problem-solving assessments provide an easy way to identify individuals who have a basic set of tools that will allow them to be an asset to the organization. Setting up such a program can often be less complicated than developing an assessment program that is tied to a specific job. There are tons of good quality, off-the-shelf assessments that have been created with the specific goal of measuring general constructs. These can be plugged right in, with less up front work than may be required to create a job specific test battery.

The idea of hire-for-potential, train-for-success is an attractive proposition in a tight labor market.

Reason 4: Assessment can support onboarding and development

Assessment is also a valuable tool because it can provide a good initial picture of an applicant’s developmental needs.

Most assessment providers have the ability to create an initial development report based on the results of an applicant’s pre-employment assessment. Even if there are few applicants for a specific job, using an assessment as part of the hiring process helps the new hire hit the ground running. Assessment data can help with onboarding by allowing the new hire’s manager to have a good idea of developmental needs on day one. This information can be used to create an initial development plan and to provide a baseline for future performance management and development activities. Such a strategy can also have a positive impact on things such as commitment, satisfaction, and turnover. Research has demonstrated that good experiences during the onboarding period can have a positive impact on each of these important outcomes.

Reason 5: Assessment can help the organization to better understand itself

You can’t expect strong results from the use of assessment without a clear understanding of what needs to be assessed, and why. Usually, this involves spending some time to take a look at both personal and organizational factors that must be addressed using assessment. More sophisticated implementations involve the ongoing use and development of an organizational competency model. Such a model allows the organization to define the key elements required for success across all jobs as well as for each specific job.

While this is not always easy, taking the time to break jobs down into the components required for success provides the organization with important insights about what is required for success. This information is valuable for hiring, but it also has value for other important processes such as development, training, and succession planning. Going through the effort required to anchor a good assessment program has value because it helps the organization learn about itself. This learning extends beyond the ability to identify key determinants of success. Using assessment properly requires a good bit of learning and experience. Why shy away from assessment during a time that can provide valuable hands-on experience? By choosing to use assessment in good times and bad, companies can gain experience that will benefit them in the long run.

No matter how many applicants there are for a given position, hiring the right one should be the result of an informed decision making process. The information used to support this process should be directly related to the various things required for success at the job and the organization. This is a universal truth. Those organizations that begin to cultivate this mindset will realize a significant long-term advantage that extends past the value of good hiring, into the realm of developing, managing, and retaining talent.

Like white? Then interior decorating is for you. (Too easy. Everyone knows white goes with everything.)

How about if your favorite colors happen to be black and red and orange? Maybe you just really like Halloween. Otherwise, you are “The Evaluator,” says a press release from CareerBuilder (profile; site), which just added a color wheel (parked on the old CareerPath.com website) to help jobseekers better assess their personality.

Before we get scolded for making light of a serious assessment tool let us note that the Color Career Counselor has been scientifically vetted with the results published in the North American Journal of Psychology. You can read the paper here, but fair warning: it’s full of the kind of statistical analysis we avoided in college.

The CareerBuilder service is powered by the Dewey Color System, which uses color selection to determine your personality type and traits. These then suggest certain types of careers that others with smilar personalties have found rewarding and successful or which typically attract people like you.

Dewey offers its colors-based assessment to companies as a pre-screening tool, touting the Dewey Employee Predictor as “the world’s first nonlanguage-based test that defines occupational interest, workplace behavior patterns, styles, traits, and temperaments, as well as an additional 16 personality factors.”

Invented by Dewey Sadka, a veteran recruiter who founded and ran his own employment agency, Temp Force, he’s written several books on the use of color in personality assessment and employment. Sadka’s latest book is on using colors to find love. His clients include Georgia-Pacific, Honeywell and SHRM. And now, CareerBuilder.

Taking the test takes only a few minutes. And we will say this, the analysis was right in more ways than we would have thought. It helps if your monitor is properly calibrated. We couldn’t see much difference between indigo and blue.

Now about that black and orange and red thing. According to the CareerBuilder press release, those who picked those colors seek “the most efficient way to accomplish tasks. Using facts, compiled from past mistakes and successes, you deliver strong opinions with valuable perspectives that maximize the bottom-line.”

And the favorite color for that bottom-line presumably would be black.

In this article, I praised the progress being made in understanding the value of quality assessment products in the modern hiring process.

I strongly believe that the words I wrote in 2006 are as relevant today as they ever were; in fact, they are more relevant now than ever. The days of the big job boards and their keyword matching strategies are numbered.

A lot has happened in the world of online recruitment in the last decade. The focus of most of the evolution has been on candidate quality and on increasing accuracy. This is in opposition to the focus a decade ago, which seemed to be on quickly delivering a large volume of candidates. What we have learned since then is that without a way to quickly evaluate candidate quality, we end up saddled with information overload. So, the objective for recruiting products is shifting to one that is focused on the alignment of personal characteristics with those that are required for performance on the job. This is a good thing. This movement has definitely been aided by the choice of several leading career portals to embed candidate quality into their bag of tricks.

A slice of the content of the original article from 2006 appears below. It has been updated to include a few examples of a few career portals that are now using assessment to help them better serve their clients (both job seekers and employers) by providing them with a way to quickly focus on candidates who have what it takes. Before we get started, I want to offer a few disclaimers here.

1. I am not saying that using assessment as part of the job search and match process provides all the data needed to make a good hiring decision. Rather, I am saying that it provides an excellent way to address the idea of “garbage in, garbage out.” By creating an applicant pool of individuals who are at least qualified and motivated, we are taking a very important first step towards ensuring systematic quality in the hiring process. I think of it as pre-screening that comes before pre-screening because it provides an applicant pool that will give one much better odds of making a successful hire.
2. In this article I am going against my policy of not ever naming the names of any vendors who provide the type of products discussed. Please note that none of the vendors discussed here have been included due to anything other than my personal opinion that they are good examples of the trend I am documenting. I was not compensated by them, nor does my inclusion of them in this article mean they are the perfect solution for your problems.
3. Although I am an expert in this space, I do not know everything, and there is definitely a chance that I failed to mention a company that provides the exact type of service I have described. I encourage someone from such companies (or a fan of one of these companies) to contact me to let me know what you are doing.

Below are some relevant highlights from my original article.

Folks are finally starting to get the idea that assessments are but one component of a broader process designed to help hiring professionals make systematic predictions that result in good hiring decisions. There’s much to be gained from this process-oriented approach, so it has been great to see assessments being used to help at various phases of the hiring process.

Despite the continued integration of assessment, there’s one area in which assessment has been underused. This area is the use of assessment tools during the job-searching process. Specifically, the integration of assessment tools into the searching/matching component of career portals.

Traditional methods available to users of career portals for locating jobs are extremely crude, consisting mostly of keyword searches or matching based on simplistic profile elements.

The fact that this basic process has endured as the standard for almost a decade now clearly reflects the quantity-over-quality focus that has reflected the marketing-centric attitude of many job boards since day one.

Blocking Out the Noise

This crude matching process results in excess noise. For applicants, noise means they’re presented with, and encouraged to apply for, a ton of jobs for which they’re not qualified. For hiring professionals, noise means no mechanism to assist them in making an initial high-level determination of applicant quality.

Noise makes it difficult to make good hiring decisions because it greatly increases the chance to make systematic errors.

The good news is that I am starting to see a shift in focus that will go a long way toward a reduction in noise and an increase in the ability to differentiate candidates based on various quality factors.

This shift involves the use of assessment tools as an integral part of the services provided by career portals. The use of assessment to help fill the hiring funnel with applicants who have the attributes required for success has numerous advantages, all of which are explained by simple probability theory.

Think about it: the more qualified the individuals in your applicant pool, the better chance you have of hiring someone who has what it takes. Over the past few years, I have learned of an increasing number of companies who are seeking to change things by integrating assessment into the matching process.

The basic aspects of this include the following:

• Seekers create profiles. This part of the matching process works as part of a registration process, before a job seeker has even expressed interest in a specific position. Along with other information collected during this process, the job seeker creates a profile based on a short assessment of basic qualities such as work attitudes, personality measures, etc.
• Corporate users define high-level requirements. This part of the matching process requires hiring personnel to create a template of the basic things it takes to do the job well. This often includes standard things such as skills and experience but also includes qualities often measured by assessments (personality factors, dimensions of fit, work values, etc).
• Noise is squelched. When a job seeker searches for a job using the profile he/she has created, results include jobs for which they are suited based on the compatibility between their profiles and those of available jobs.

There are many twists on this formula but the basic idea is the same: deliver an applicant pool that has less noise and makes the recruiter’s job easier. This is the start to ensuring quality hires because it’s sure hard to hire superstars if they aren’t even in your applicant pool to begin with.

Admittedly, my description of this process is overly simplistic. In job matching, just as with making hiring decisions, assessment data should serve as one of many data points that are used to identify the best candidate for the job. Truly effective matching systems will use parameters that include other key determinants of success such as experience, knowledge, and skills.

Weighing the Pros and Cons

Overall, the use of assessment in the matching process offers the following five advantages:

• Reduces noise. Allows filtering based on actual job-related parameters.
• Helps determine corporate fit. This process is a good way to find and hire applicants who fit a company culture.
• Increases quality. Allows insight into applicant’s ability to deliver what it takes for success.
• Educates job seekers. Allows applicants some insight about themselves and how suited they are for a particular job or career.
• Results in branding. It can provide a good applicant experience, leading to support for employment brands.

Of course, there are potential drawbacks to the use of assessment as part of the matching process. These include the following five issues:

• Learning curve. Users must learn to understand how to define jobs in terms of relative importance of the parameters used for matching.
• Accuracy. It is critical to match the templates against which applicants are matched to accurate reflections of the job.
• Uncertainty. Predicting performance is a difficult task, to say the least. Assessment-assisted matching is not a panacea and does not eliminate possibility that mistakes will happen.
• Quality control. To be effective, assessments must be created using the proper methodology; substandard assessment products will produce substandard results.
• Legality. While the use of assessments in matching is not illegal, it does require awareness of some potential legal ramifications (see original article for full coverage of legal issues related to the type of products discussed in this article).

So who is actually using some sort of assessment as part of a career portal or job board? I believe the companies below are all doing great things to move us forward toward an era of embedded assessments that can help reduce the noise associated with finding and applying for jobs online.

Climber: Climber.com uses some really cool technology to create a comprehensive applicant profile that includes an assessment of work values. The complete applicant profile is then compared to a profile created by employers making job postings. Part of this process includes an employer work-values profile that helps capture the values of the organization in order to help ensure that users (both applicants and employers) are matched based on a mutual “fit.” Climber goes beyond the norm for this type of exercise by using sophisticated data analysis techniques and AI in order to help optimize the search/match process for its members. It is truly on the cutting edge when it comes to delivering candidate quality via tight searching/matching through the use of sophisticated data analysis techniques and AI.

Jobfox: Provides applicants with the opportunity to take a short personality assessment once they have registered. While the results of this assessment are not currently used to help match job seekers and employers, Jobfox members are able to see the results of their assessment and use them for self-understanding and developmental purposes. Jobfox also has a very nice skills matching technology in which applicants take the time to create a skills profile that is then matched against the skills profile that is created for each position that is posted by organizations using Jobfox. The result is a match score that provides candidates with an idea of which of the jobs available on Jobfox are a good fit for their skills and experience.

Monster: Monster has partnered with DDI, a leading assessment firm, to provide its clients with the ability to add assessments to any job posting. This product is known as the Monster Performance Assessment or MPA. The ability to provide the MPA for most job postings is made possible by DDI’s extensive experience with assessment tools as well as the data it has collected over the past several decades. The MPA package represents a groundbreaking offering, as it is the first time assessments have been sold transactionally as part of the process of posting job openings. The MPA does not use assessment to help candidates identify jobs for which they are matched; rather, it provides recruiters with much richer data about a candidate’s qualifications for the position of interest. This allows recruiters to quickly shortlist candidates based on a set of standardized, job related data. While there are still some details being ironed out, Monster’s approach represents an interesting and potentially effective way to help deliver quality. Monster is also the only one of the mega-boards that has been bold enough to step out with an assessment-related solution.

eBullpen: eBullpen is a job board that uses a personality assessment as part of the process used to match job seekers with available openings. eBullpen has been working on its process for several years now and provides a legally sound, useful approach. This approach asks companies posting jobs to outline the personality traits, activities, and requirements for the job. Job seekers provide data on their personality traits, work preferences, and qualifications. Matching alogrythms are then used to help match job seekers with available openings.

Each of these four companies are stepping out into uncharted territory and is doing so with a slightly different approach. The specifics of these approaches aside, the big picture is what matters the most here.

This big picture is the idea that each of these folks provide a way for job posters to break down a job into a series of key data points which are then matched to a series of related data points in a profile or assessment snapshot created by job seekers.

Detractors have often lobbied against this kind of matching, saying that no candidate (or company for that matter) is interested in taking the time required to create the profiles needed to fuel this kind of process. I firmly disagree. Good hiring takes an investment on the part of both the job seeker and the employer, and good investment often takes a bit of effort. The old adage, “you get what you paid for” along with my personal favorite “garbage in, garbage out” both apply to this situation. I am confident that as we travel further down the road of embedded assessment, the data will bear out the fact that the small amount of time taken to fuel the matching process will prove to be time well spent

Despite this, within the next five years I expect to see career portals offering an increasing amount of value by beginning to change the focus from quantity to quality, with assessments as one of the core elements of this switch.

I think all stakeholders involved in the hiring process, including applicants, stand to benefit from this trend.

Total revenue grew 9% to $354 million, from $324 million in the comparable quarter of 2007, boosted by a favorable exchange rate. Even without the benefit of the exchange rate Monster grew revenue by 4 percent. Wall Street analysts had estimated revenues would come in at $361 million.

International sales fueled the company’s growth during the quarter, as it has for the past year. While revenue from North American operations fell by $10 million during the quarter to $164 million. But sales elsewhere in the world jumped 34 percent (23 percent when you exclude the effect of currency exchange rates). International sales now account for 44 percent of company revenue.

Much of the financial news was expected by stock traders. Monster shares closed at $17.74, down 2.4 percent on the day, and rose slightly in after hours trading, when the numbers were released.

The company also reported settling a federal class action suit brought on behalf of shareholders who claimed they were mislead by the company’s practice of backdating stock options. That settlement and one previously announced resolves most of the shareholder litigation.

The Trovix acquisition was more of a surprise. Monster CEO Sal Iannuzzi called the purchase “a game-changing development.” “(It) complements nicely our ongoing investments in product, sales, and customer service, as well as the extension of our business into new areas and geographies,” he added.

Trovix launched as an ATS vendor in 2002, implementing artificial intelligence to analyze reqs and candidate resumes and match them in ranked order. The matching technology was refined over the years and in 2007 Trovix introduced its own jobboard where standard job postings and resumes and jobseeker information were matched.

In the announcement of the deal, Monster explained its rational for acquiring Trovix: “The combination of Monster and Trovix enhances Monster’s value proposition to employers by adding speed and efficiencies to the recruiting process, resulting in an improved return on investment for employers. The technology will also benefit job seekers, who will receive more relevant, targeted job postings.”

“The implementation of this technology will allow Monster to provide unparalleled match capabilities, taking us beyond keyword search into contextual search,” says Darko Dejanovic, Monster’s Global Chief Information Officer and Head of Product.

At least for now, Monster will continue to support existing Trovix customers.

Monster has also announced the acquisition of Armees.com, a French military site similar to Monster’s Military.com. No price was announced.

Finally, Monster said it has entered into arrangements with vendors Cornerstone OnDemand, a provider of on-demand, E-learning and training services, and HireRight (profile; site) an on-demand employment screening solutions provider.

Cornerstone will provide online courses in general business, desktop applications, and IT topic. HireRight will offer employers the opportunity to purchase background screening services as part of its current candidate management experience on Monster.com.

Imagine advertising for superheroes. There are a dozen steroid-pumped, ego-centric applicants sitting in your waiting room wearing masks, capes, and tights. Each hero claims to have saved the world at one time or another. You hire three of them. Six months later, how do you evaluate your quality of hire?

Evaluating quality of hire requires looking at performance in a different way. It requires mentally separating the “how” from “what.” The “how” represents what the superhero says or does and “what” represents the outcome. Here is the hard part to accept: evaluating quality of hire depends almost entirely on evaluating “how” the hero performed the job, not the outcome. Regardless of opinions to the contrary, “how” is the only part of the job under the hero’s control. It is the only thing separating one hero from another.

Here is an example that may explain this idea.

It’s a particularly bad time on earth. Asteroid showers are occurring periodically, keeping the super-heroes busy. When he was on duty, Clock-Man reacted by turning back time. On her shift, Wonder Woman pulled the asteroids into new orbits with her lasso. And when it was his turn, Superman flew faster than a speeding bullet, smashing them into smithereens. The “what” was the same for all three: reversing time. Using lassos and brute force were all examples of “how.”

What about evaluating the quality of hire? Let’s look closer.

Clock-Man’s action trapped the world into a year-long time loop. The world kept rewinding and playing back. The asteroids never hit, but the world missed celebrating Thanksgiving and Christmas, causing turkey famers all over the land to go bankrupt. Superman decided to stop first at Starbucks. When he finally finished, the asteroids were too close to be completely destroyed and one piece broke off, annihilating New Jersey (starting a lively argument about whether New Jersey was a fair trade for a decent Venti Mocha Latte).

Wonder Woman was shopping for glassware and misplaced her invisible plane among the fine crystal. When she finally found it, was able to lasso the largest asteroid, but a small one evaporated Paris Hilton and a small platoon of paparazzi (although no one except Mr. and Mrs. Hilton seemed to care). Now can you evaluate quality of hire?

Evaluating quality of hire is based on the same elements as determining which applicant to hire…you have to decide beforehand “how” a job to be done. Looking only at results can confuse performance because there are so many other things that can affect them. Superman might have been successful if he was motivated. Clock-Man should have thought through the long-term consequences of time-tinkering. Wonder Woman might have been more successful if she would have recognized the problems associated with finding an invisible glass plane in a clear-glass factory.

Human performance always has three components: 1) an antecedent or event; 2) the candidates’ response or behavior; and, 3) the consequence or result. Folks call these the A-B-C of performance. The antecedent and consequence are the “whats” (i.e., the results). The candidate’s behavior is the “how” (i.e., what the employee said or did when confronted with the situation). “How” is what we use to define job requirements, select and promote employees, and evaluate quality of hire.

You can think of every job as having standards for motivation, organization, analytical thought, learning, and behaving. For sales jobs these might include competitive drive, time and territory management, sales development strategies, learning new products, making presentations, and so forth. Management jobs might include the motivation to direct and develop subordinates (instead of doing it yourself), achieving objectives, solving problems, managing the marketplace, and coaching skills. Jobs always have cognitive components and behavioral components.

The key to understanding “hows” is knowing which behaviors vary with the job holder, which are necessary for successful job performance and which are associated with failure.

Why don’t we more often use “hows” to evaluate employee and applicant quality?

1. “Hows” often occur hours, days, weeks, or even months before we see results. We either forget or overlook them.
2. Results are usually in-your-face singular events that command attention, whereas “hows” are more subtle and might occur together in clusters.
3. We look at results and jump to conclusions about “hows”, often taking them for granted.
4. Some people take credit for other’s “hows.”

I’m sure you can think of more, but people seem to have an intuitive understanding that how’s are important. But because most folks are not measurement experts, they get them confused. For example, let’s look at a sample list of recommended action verbs (i.e., hows) taken from a career-services center. They recommend using words like administered, analyzed, attained, chaired, contracted, consolidated, coordinated, developed, and strengthened. Unfortunately, if you are screening resumes or interviewing applicants, the only verb that even closely resembles a candidate-centered how is “analyzed” …the rest invite assumptions.

People are wired internally to make fast decisions based on little data. While this might be a good survival strategy, it leads recruiters and hiring managers to make huge assumptions about candidate skills. Negative information, for example, (e.g., a typo in a resume) leads us to assume the candidate is sloppy and inept. Positive information (e.g., high sales dollars) leads us to assume the candidate is highly skilled. A successful recruiter who knows how to identify and evaluate candidate “hows” will both recruit better candidates and be able to better evaluate quality of hire.

When we see a candidate who meets a large number of our pre-existing conditions for employment (i.e., a candidate who has gone to a school our hiring manager likes; has worked at a couple of well-respected companies; or has written the right key words on his resume), we have already hired him in our minds.

Interviews are examples of how easy it is to abandon the tools of objectivity, the scientific method, logic, and the rules of evidence, for our “gut” or for “chemistry.”

While there is considerable evidence showing that testing candidates is far more likely to predict successful performance, we still rely almost exclusively on interviews. Though numerous researchers have pointed out the need to gather a variety of data about a candidate, we generally settle for an application form and an interview.

Why are we so resistant to testing and other more objective sources of data?

Perhaps it is because our expectations, preconceptions, and prior beliefs pretty much always influence our interpretation of new information. Experiments conducted over and over have shown that we see what we expect to see and conclude what we expect to conclude.

Tom Gilovich, a Cornell University psychologist, writes:

“Information that is consistent with our pre-existing beliefs is often accepted at face value, whereas evidence that contradicts them is critically scrutinized and discounted.”

I was recently helping a hiring manager evaluate his staff for promotion. We carefully determined success criteria and agreed that the candidates had to meet those criteria for consideration. In fact, we spent quite a bit of time validating the criteria and ensuring that others agreed with them. We then tested his entire staff using highly validated, widely accepted tests of ability and potential.

As you might expect, one of his favorite people did not do very well on any of the tests. He struggled with what to do and finally decided that his own judgment was more valuable than the tests and promoted her. Six months later she was not performing, was not happy, and he was now faced with the task of demoting her or letting her go.

This is just one example of some general truths. If we are presented with evidence from a test that a person we really like is not very good at whatever skill we are looking for, we say that he must have had a bad test day or we find some other excuse to downplay the tests results because we want to hire him.

On the other hand, if the person had not gone to the right schools and not worked at the right companies, or had displeased us in some way, we would be more likely to accept the test results as accurate.

There are countless examples of how we deceive ourselves in the process of interviewing and screening candidates. We ask leading questions to elicit the responses we want: “You have made presentations to senior management, haven’ t you?” We ask references the same kinds of leading questions.

It is not that we don’t examine information critically. In fact, experiments have shown that we look at all the evidence quite carefully, but we subtly massage it to make it support our preconceived idea or wish.

If evidence seems negative of our desire, we find excuses for why the information is bad or we lower it in our priorities for making a decision. We do just the opposite for favorable information.

We also will find data to validate our choices later on. If a person is successful, we will tend to attribute that to our superior interviewing skills, but if they fail we will find other reasons. Managers and recruiters are expert at the art of scapegoating their poor hiring decisions. What is most interesting, is how often someone removed from the process predicts the end result well before it happens because they see things more clearly and do not suffer the same preconceptions.

The bottom line is that assumptions, beliefs, and interviews are very poor tools for selecting people for specific jobs. It is almost impossible to apply objectivity to the interview process or to rid ourselves of deeply held (and mostly unconscious) beliefs.

Here are three things you can do to make you more effective as a selector:

1. Become aware. While we cannot prevent our preconceptions from clouding our judgment, we can apply corrective measures. We can develop criteria for jobs that are based on competencies, not on beliefs that cannot be validated or objectively supported. We can apply the scientific method to the recruiting process, just as we do to most other processes. I highly recommend How We Know What Isn’t So by Thomas Gilovich at Cornell, and Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart by Ian Ayres. They are eye-openers to how easily we are duped and misled by seemingly objective evidence and by our own human nature.
2. Learn that superficial, circumstantial evidence is probably misleading and often wrong. Every court of law has developed elaborate rules of evidence to ensure that they get as accurate and well-rounded view of a situation as possible. Even with all of those rules and procedures, many innocent people still get convicted. Finding objective criteria and evidence takes time and a willingness to seek it out.
3. Use objective tools such as validated tests and multi-rater feedback. By starting with one or two well-known tools, we can refine and hone them to our exact needs until they are excellent at predicting success. Proctor and Gamble has been doing this for more than two decades with remarkable success.

When it is essential that people learn rapidly and perform at superior levels, you need objective and repeatable ways to judge candidates. No scientist would rely on interviews, feelings, or opinions to judge a scientific experiment. Neither should we in judging a candidate.

Article and research by Charles Handler and Mark C. Healy

For the last five years, Rocket-Hire has surveyed talent-management professionals who use Web-based screening and assessment products to enhance and quantify their hiring processes. Our goal for this research is to document trends in the usage of online screening and assessment tools in order to help provide a clear understanding of the popularity of these tools and their perceived positive and negative attributes.

Late last year, we again asked recruiters, HR staff, entrepreneurs, and hiring managers to tell us about their use and opinions of typical hiring instruments. This article highlights our major findings.

Today’s report is just a brief summary of the results of this study. For more data and an in-depth analysis of these trends, check out your May issue of the print publication, the Journal of Corporate Recruiting Leadership.

Overall, our findings from this year, and the trends that emerge over the five-year life span of this survey, continue to be positive. We had the largest number of respondents yet this year (141 recruitment and hiring professionals completed the survey, just a tad above last year’s 136).

The increase in use of screening and assessment tools and the positive benefits of these efforts continue to notate a strong trend toward their popularity and value.

On the negative side, results clearly indicate a continued lack of proper evaluation for these tools, which is even more discouraging when one considers that lack of understanding of the benefits of screening and assessment is still a primary reason for the failure to adopt them. Thus, the need to demonstrate the value of assessment by including evaluation as a core part of implementation is crystal clear and, in our opinion, remains the No. 1 challenge related to the use of assessment.

Now on to a more detailed review of our results.

Use of Applicant Tracking Systems

As with previous surveys, a majority of respondents employ an applicant tracking system. Specifically, 79% already have or will soon have one deployed, or are considering installation. As can be seen in the table below, no matter what the hiring demand, typical ERE readers are familiar with using an ATS.

Table 1. Use of Applicant Tracking Systems by Hiring Demand

Use of Prescreening Tools

About 57% use online prescreening to sort through their job applicants. We defined prescreening tools as:

Tools that gather information about, or ask candidates to respond to questions about their experience, skills, and qualifications in order to identify if they meet minimum job requirements. These tools are typically used early on in the staffing process.

The table below captures the usage rates of each of the most prevalent forms of screening.

Table 2. Usage rates of common prescreening tools

Common tools in use include the popular qualifications screen, technical skill certifications, and analyses of “fit” with company culture. Every year, it seems that more organizations are using prescreening as part of their online recruiting strategy. But the breadth of this practice varies, with some companies using these tools for all jobs while some use it for one or two alone.

Table 3. Extent of Prescreening Deployment

Use of Assessment Tools

In contrast to screening tools, online assessment tools were defined as:

Scientifically based screening tools that look more deeply into a candidate’s abilities, interests, and skills. These tools include personality measures, cognitive tests (i.e., verbal and quantitative skills), situational judgment tests, job simulations, etc. These tools are typically used for a more in-depth evaluation later on in the staffing process.

Underscoring the growing increase in the popularity of these tools, most common forms of online assessment showed an increase in rate of adoption over previous years, as comparisons with 2002 reveal.

Table 4. Usage Rates of Common Assessment Tools (2007 vs. 2002)

Specific skills and knowledge certifications and personality inventories are common assessment strategies once the pool of one’s qualified candidates has been reduced. Indicators of cultural “fit” and various cognitive ability measures are also widely used.

For another angle on the extent of assessment implementation, we asked respondents to indicate the level of jobs for which assessment is used for evaluating candidates.

Table 5. Job Level of Assessment Implementation by Job Level

2007 appears to be the first year that our data show a general pattern of use across all levels of an organization. Nonetheless, executive-level openings tend to entail more personal and direct methods of selection and placement, and were less likely to involve the use of any online tools.

Effectiveness of Screening and Assessment Tools

This year, 65% of prescreen users and 77% of assessment users felt their tools added value to their organization. In contrast, a full 21% of prescreen users and 10% of assessment users felt these tools did not add value. But this is difficult to judge, especially if one does not take the necessary steps to adequately answer the question. In this case, only 27% of users of prescreening tools and 36% of assessment users collected metrics to judge the quality of their interventions. Moreover, users of metrics tend to report success with prescreening and assessment.

Table 6. Perceived Effectiveness of Prescreening Tools

Table 7. Perceived Effectiveness of Assessment

Clearly, collecting metrics helps to answer questions about effectiveness and, interestingly, provide a positive response to these questions.

The Future of Online Screening and Assessment

Of those companies not currently using screening or assessment instruments, 42% feel they will implement one or more in the future. For them, Table 8 summarizes the sorts of tools under consideration.

Table 8. Screening and Assessment Tools under Consideration

As with the tools current users are seeking to use more of (or expanding to include in their hiring systems), qualifications screening, assessment of cultural fit, skill/knowledge assessments, and personality inventories dominate interest. Still, the mystery surrounding the value provided by these tools, as well as a prevalent intimidation factor, keeps companies from jumping in.

Obstacles to the Use of Online Screening

As usual, we asked users of online hiring technology (as well as non-users), what they perceived to be the biggest obstacle to the adoption of prescreening and assessment in their organization. Again, we received nearly identical responses to previous surveys.

Table 9. Single Biggest Perceived Obstacle to Deployment of Online Tools.

While 18% of users feel there are no obstacles at this time, many hirers feel that one or more roadblocks prevent prescreening and assessment from being integrated into the staffing process. These reasons mostly reflect a lack of knowledge of the benefits of more in-depth hiring techniques. Moreover, there appears to be a general lack of knowledge of safeguards and metrics designed to detect problems when candidate evaluation systems are in place, even among regular users of assessment.

But do these worries represent valid concerns, or is it simply fear of the unknown? Arguably, when assessment tools are properly matched to the demands of the open position, the only relevant concern is applicant identity (e.g., the stereotypical “smart older sister” taking a math test for an applicant) and test content security. But candidate enhancement of qualifications is common on resumes, in interviews, and for some personality traits/scales. Since many of the same concerns exist for paper-based processes, this should not limit any one organization from deploying Web-based hiring technology. Overall, most companies may legitimately cite few or no substantive reasons to limit their use of online recruitment and hiring techniques.

Summary

This year’s results let us know what’s really going on, and it confirmed what most of our previous surveys have found:

• Having an ATS installed is now nothing particularly innovative, especially if you work in a medium or large organization. For some, it’s hard to even imagine the old stacks of resumes and bulging file folders of applicant information.
• Prescreening and assessment are continuing their gradual penetration into the mainstream of recruitment and hiring.
• Qualifications screening, personality inventories, and skill and technical certifications continue to be the most popular online assessment tools. Assessments of cognitive abilities as well as fit with the company culture have expanded their footprint as well.
• Respondents report a lack of understanding, weak budgets, or a general lack of support for online tools as their primary obstacles to adoption or greater use of modern prescreening and assessment technology.
• Those organizations that formally evaluate their hiring practices tend to support the use of prescreening and assessment, but a large number of organizations are still failing to evaluate the effectiveness of their screening and assessment tools.

There clearly is a growing interest in scientifically derived hiring tools. This trend is encouraging but not particularly surprising; these numbers will continue to grow as more organizations understand the value in properly evaluating the impact of screening and assessment tools, seeing for themselves how quality hiring tools can improve a workforce.

As both authors have seen over a combined 25 years in screening and assessment consulting, skepticism about the value of these tools remains strong. We will continue to champion the use of evaluation to “close the loop” and reveal how more in-depth evaluation of candidates may positively impact the bottom line.

A word about our methodology: Respondents were evenly representative of recruiters, recruiting leaders, HR executives, business owners, and hiring managers, and represented a wide variety of organizations and hiring situations. Specifically, 59% were employed by organizations with 500 employees or more, with 34% working for a company with 5,000 or more employees. In contrast, about 20% were involved with recruiting for companies with 50 or fewer employees. In terms of employee acquisition activities, 34% hired more than 500 workers in a given year, but a majority of respondents (66%) made 500 or fewer hires per year.

The phone rings. Someone on the other end says he or she wants to build (or buy) a Web-enabled hiring test. Let’s say it will be for salespeople (generally the caller is a recruiter or HR manager, but sometimes he or she is a gopher).

After discussing the idea for a few minutes, I make a few suggestions. These always include following the ‘Guidelines’ to make sure the test is based on job requirements and business necessity and following the ‘Standards’ to make sure the test actually predicts job performance.

In almost every case, the caller is aghast at the work that needs to be done.

“All I want is a test!” they say.

“You want one that works?” I ask.

“Yes. But that’s hard!” they say.

“What’s your point?” I say.

On the other end of the phone, my keen bat senses hear muttering about me being a “jerk,” then dialing someone who will sell them the “mother of all tests”?one the vendor promises will work, regardless.

Why All the Fuss? A Test is Just a Test, Right?

The Guidelines and the Standards are not “nice to know” (i.e., limited to eggheads, legal eagles, and companies with U.S. operations). They describe how to define and evaluate job skills. That is, they first recommend test users define critical elements of the job based on job requirements and business necessity; then, they describe three ways to make sure test scores accurately predict performance (e.g., criterion, construct, and content validation).

Let’s reiterate. Step 1: Define job requirements and business necessity. Step 2: Make sure the test is predictive and stable.

Clear definition and evaluation is good for the hiring organization and good for the applicant. This principle works in all cultures and countries. So, if you plan to use a Web test, it’s a good idea to know the test actually separates qualified applicants from unqualified ones.

If you don’t get anything else, patch this into your screensaver: the only people who think it is too much work to follow best practices are people who don’t know how to do it.

But, no harm is done, right? Wrong. Highly effective hiring tests that claim they have no adverse impact or have been “validated” by the U.S. EEOC are as legitimate as the email announcing you won the lottery in Botswana.

Bad tests are really bad news for employer and applicant alike. A bad product backed by good-sounding marketing claims is still a bad product. And whether the user is in the U.S. or not, the test consumer, not the vendor, lives with the consequences of test use!

So, even if the vendor claimed his test was validated to grow hair on bald applicants, transform ugly employees into movie stars and cure morning breath, it would be your problem, not the vendor’s, to prove it.

Cause and Effect

There is a good reason why sailors advise passengers not to spit into the wind. The same is true for feces, fans, and bad tests. Eventually, even clueless test purchasers learn a weak test does not work as promised. You see a test that is not based on job requirements and business necessity, nor validated for the specific job, is designed to pass too many wrong applicants and fail too many right ones. It will show up on the job. That’s why the Guidelines and Standards are so valuable: they define exactly how to identify, qualify, and use a test that contains the least amount of error.

The bottom line is no matter how many years a person has been a recruiter; no matter how smooth his or her marketing campaign; no matter how certain he or she is about being a recruiting expert; and no matter how famous their organization, the ‘Guidelines’ and ‘Standards’ set the bar for measuring job skills.

Let’s examine how the ‘Guidelines’ and ‘Standards’ work for a sales position.

Sales Hiring 101

First, any method of separating qualified from unqualified applicants is a test. And “assessment” is just another word for “test.” We assess resumes, application forms, and applicant skills. The vast majority of organizations, unfortunately, use a two-step assessment process. Step one: use an interview to screen out most of the riff-raff. Step two: let the job screen out the rest. The two-step process explains in large part why 20% of salespeople generally produce 80% of the sales. Only riff-raff were screened out pre-hire.

Screening out riff-raff is easy. All you have to do is get to know the applicant, examine earnings statements, and dislike his or her personality. Normally, organizations screen-out 3.5 applicants to get one promising employee. On-the-job performance screens another one of two. Over time, this makes the final hiring ratio about 7 to 1. Riff-raffing is the norm and riff-raffing is expensive.

Let’s look at the cost of using the job as an assessment in terms of training, travel expenses, management coaching, and salary for six months. We’ll be conservative. One week training = $2,500; sales travel expenses = $100/day for six months or $12,000; coaching time = 15% of manager’s time or about $6,000; and, six months’ salary and benefits = about $36,000. This totals about $56,500 per salesperson (ignoring recruiting fees, lost customers, empty territories, and so forth). Bottom line? In round numbers, riff-raff assessment costs upwards of $50,000 for each lost salesperson.

Error-Free Hiring?

Mistake-free hiring is pure fiction, but doing a better job screening is not. First, you have to fully understand your specific sales job and the critical skills that separate the successful from the unsuccessful salesperson. This kind of information is seldom obvious. It does not come from generic tests, averaged scores, and calculating group norms. Generic norming is bad science. It serves as an example of wrong-headed test practices.

A trustworthy and reliable test involves in-depth understanding of critical job functions, measuring every critical skill area at least twice, doing a formal study to confirm scores predict job performance, and monitoring adverse impact. In professional terms, this is called job analysis, validation, multi-trait-multi-method assessment, adverse impact monitoring, and continuous improvement. If it sounds like a good way to do business, it is. If it also sounds like hard work, it is.

In the next few paragraphs I’ll briefly describe what to look for in a sales selection system.

Professional Job Analysis

As mentioned above, a professional job analysis does not consist of giving everyone a questionnaire and comparing top-performer scores to bottom performers. This is the first sign of buyer-beware because it makes some huge and often wrong-headed assumptions.

It assumes an equal playing field. That is, all productivity results are equivalent. New accounts, customer service, market conditions, and expanded accounts are all rolled-up into the same category: productivity. In some cases, overall performance might even be complicated by (gasp!) skillful manipulation of numbers. Separating salespeople into top and bottom producers based on sales dollars is a sure clue the analyst does not understand sales.

Suppose you are like most folks in the hiring business and you expect your test to accurately predict job performance before you commit big bucks to salary. By definition, your test should measure something that causes performance. If you give one big test to everyone without knowing explicitly what you want to evaluate, you fall into the “correlation or causation” trap. As an example, ice cream sales and shark attacks have a strong positive correlation. Does that mean shark sightings cause people to eat more gelato? That Ben and Jerry’s Chunky Monkey is a poor shark repellent? Or perhaps sharks have a seasonal business they don’t want people to know about? Homegrown questionnaires often confuse correlation with causation. Just remember: Unless water-born ice-cream is proven to attract sharks, one does not cause the other.

A good job analyst knows how to identify key skills that make the difference between successful and unsuccessful cold calling, repeat sales, strategic selling plans, customer service, and so forth. In many cases, they may involve totally opposite skills. Treating sales production as a discreet measurement point is like putting fruit salad in a blender, pressing the annihilate button, and testing the puree for peaches. A professional job analyst knows key information can only come from people doing the job, not from supervisors or aggregated production data.

Let’s say the analyst has done his or her homework. Now what? The hiring manager does not have weeks or months to evaluate applicant skills. Unless the hiring manager uses the hire-and-hope strategy, sales skills have to be evaluated in minutes or hours. If we have done our job right, we will know the mini steps that lead to maxi results.

Bottom line? If the analyst asks you to lump producers into groups and gives them all the same test, you are about to see your money pour out the door.

Does the Test, Test?

The only test that is worth anything is one that works for your job in your company, not one that worked for the company across the street, or a job with the same title, or matches a nationwide norm, or even a company in the same industry. It has to work for you.

Sometimes a validity study can be transported from one job to another, but that is only if you know for certain the two jobs are essentially the same. But if the market is different, the company environment is different, products and services are different, customers are different, or sales cycles are different, then how can any reasonable person claim XYZ scores predict cold calling, customer service, or sales expansion for your position based on one that is entirely unknown? Doesn’t that seem a little far-fetched to you?

The only time you can trust that another test will work best for your organization is to compare the job analysis from the other test to the job analysis for your job. If the two jobs are essentially the same, then use it; if not, you “pays your money and takes your chances.”

Give a generic personality test to salespeople and see what shakes out? Get ready to see a great big pile of belly-button lint.

Our Test Does Not Discriminate

In the U.S., at least, large organizations and Federal contractors are not supposed to reject qualified applicants based on age, gender, race, and so forth. This is called discrimination; but there is something else called adverse impact. What does adverse impact have to do with discrimination?

The legal definitions have subtle overlap, but for the purposes of this article, let’s assume discrimination generally means that an organization intentionally discriminates against certain kinds of job-qualified people?in hiring, promoting, training, and so forth. While adverse impact generally means the hiring system, even though it is job-related and professionally validated, unintentionally discriminates. In lay terms, think of discrimination as intentional and adverse impact as unintentional. For any better definition, see your local labor-law attorney to explain the details.

I consider discrimination unethical. Everyone deserves a chance to work in a job for which he or she is qualified. But here is where things get complicated. Government agencies examine discrimination at the group-level. Hiring managers don’t care much about group performance. They care about individual performance.

This raises a problem that all hiring professionals need to consider. By way of example, suppose 200 people apply for a job. One hundred are Lilliputians and 100 are Yahoos. At the group level, 70% of the Lilliputians are hired, while only 40% of the Yahoos make the grade. At the individual level, there are quite a few Lilliputians who are miserable workers, just as there are quite a few Yahoos who are top performers.

From the organization’s viewpoint, they only hired job-qualified people. From the government’s viewpoint the company discriminated against the Yahoos.

Who’s right? It’s hard to tell, so the government examines the organization’s:

• Professionally developed job analysis (to show hiring tools are based on job requirements and business necessity)
• Professionally conducted validation study (to show hiring tests and interviews accurately and consistently predict performance)
• Pass and fail results for Yahoos and Lilliputians at each step of the hiring process
• Proactive efforts to develop tests with less adverse impact on Yahoos

As long as the company has done its homework and followed generally accepted hiring practices as outlined in the “Uniform Guidelines” and “Standards,” it is not in trouble and will have hired all the best and most diverse applicants.

So what’s the problem? Some vendors claim their tests have no adverse impact. But research consistently shows hiring tests for jobs requiring problem-solving ability almost always does have an adverse impact when examined on a group level. Competent test vendors know this. Incompetent ones don’t.

Automated Resume Screens

What recruiter or hiring manager has not seen a brilliant resume developed by a blatantly unqualified candidate? And what about the marginal resume presented by a remarkable applicant? At best, a resume includes Kodak-moments recalled by the resume writer. At worst, a resume is an exercise in creative fiction.

Think about it. Every applicant is motivated to write just enough words to garner an interview. While every hiring manager wants to find someone who was an exceptional performer in the exact same job at another company doing the exact same work. Generality goals meet specificity objectives.

Sophisticated applicants know how to pepper the resume with keywords and qualifications that may be fact or fiction; different hiring manager’s screen resumes using totally different criteria for the same job; and, everyone makes massive inferences based on snippets of data. So. Tell me again. Other than keeping a few programmers in work, what is the benefit of automating resume searches?

Back to the Beginning

So here we are, back at the beginning. Tests are abundant. And if all you want to know is a score, anyone test will do. Good tests, however, ones that accurately predict job performance, are rare. You can trust a good test to produce good employees. You can tell the difference by following a few guidelines.

Avoid vendors that emphasize their non-discrimination aspects, “legality” or industry-wide applications. Assuming their claim is accurate (and I have yet to see one what was) users are responsible for their own test use. Vendors are off the hook.

Avoid vendors that want to give their test to two groups of producers and use the results to predict job performance. These represent bad science. Scientifically, this kind of study can only show whether the two groups are different, but it does not tell you why. And it does not tell you about individuals within the groups.

Avoid tests that are based on self-reports. Self-reported answers can be faked. They cannot be validated by outside sources. Self-reported tests are similar to resumes. They represent things the test-taker wants you to know about him or her. Making decisions about hard skills based on self-reported data requires a huge leap of faith that is generally wrong half the time.

Ask the vendor for a report showing he followed the ‘Guidelines’ and ‘Standards.’ This is your only assurance the test will be job related, based on business necessity and accurately predict job performance.

Web-based testing is in the same category as medicine was 100 years ago when heroin was good for you; there was no such thing as anesthesia; injections were unavailable; radioactive water cleared the mind; opium was a relaxation agent; blood-letting was commonplace; linseed, mustard, and soap were used as cure for infection; and sugar of lead was a common treatment for diabetes.

Let’s all work hard to move hiring into the 21st Century.

Every year, I take a few minutes to reflect on the things I have noticed while working with both producers and consumers of screening and assessment tools. Overall, I am very encouraged by what I have been seeing. The market for screening and assessment tools continues to grow. This makes me extremely happy because we I/O psychologists know the value that is to be had via the use of quality assessment tools.

The science geek in me is also very happy to see strong investment in innovation. I am really pleased to see the ways in which quality content is being combined with technology to collect the mountains of data that are required to uncover underlying truths about the relationship between human traits and job performance.

One of the most valuable means of insight into assessment trends over the past five years has been the annual ERE/Rocket-Hire Screening and Assessment Usage Survey. If you have not taken the time to complete this survey yet, please consider taking five minutes to help us out. Your input really does make a difference, and we greatly appreciate your time. A link to the survey is provided at the end of this article.

The major theme for 2008 will be increased integration of assessment into carefully designed products. Much of the rationale behind these products is the reduction of the work required to implement assessment via a combination of data and technology that results in a turnkey product that is both relevant and easy to use. This trend is going to continue to increase the accessibility of assessment and lower barriers to entry that have long been limiting the use of assessments. A quick look at the trends outlined below clearly demonstrates support for this point. In 2008, I believe we will see the following trends continue to unfold:

1. Continued Interest and Use in Assessment. The market for assessment tools will continue to increase as it has every year for the past decade. This will be the result of a number of factors, many of which are discussed in more detail below. In my mind, however, the number one reason for increased adoption is still the fact that properly used screening and assessment is one of the most important ways to increase the quality of hiring decisions.
2. More Vendors to Choose From. As interest increases, so shall the number of options available to consumers. For the most part, this is a good thing, as the bulk of new companies and offerings are being created based on solid best practices. As always, there will continue to be less sophisticated or technically-sound options available. With so many options to choose from, extracting the most value out of assessment tools will continue to require a solid game plan and due diligence as the foundation of the vendor selection process.
3. Increase in Acquisitions for Content. As companies continue to work toward developing products and increasing use of assessment tools, many are discovering that it is easier to purchase content that has a strong history behind it than it is to create their own. This has also been playing out as existing assessment companies continue to purchase smaller companies in order to increase their product range. Recruitment Process Outsourcing (RPO) is also a huge driver of this trend. RPO will continue to be a major force in the future of hiring, and assessment should end up as a key part of the better RPO offerings.
4. Increased Integration and Packaging of Assessment into Off-the-Shelf Products. Directly related to the trends mentioned above, there has been a continued push towards the development of product suites that provide a hiring process into which assessment is embedded. The goal is to create a more turnkey product that can be lightly customized and quickly implemented. The more quality assessment that is baked into a well thought-out process that can be implemented relatively easily, the more quickly assessment can begin to return value. This equation also lessens the need for professional services and makes the product a bit more “foolproof.” The more turnkey the product, the less assurance one has that it is “dialed in” to a particular position or job. The decision to use a more turnkey product as opposed to a more customized approach should be treated as a business decision driven by the contingencies of one’s particular situation.
5. Increased Attention to Products/Sales into Vertical Markets. Again, as the trends noted above unfold, we are seeing an effort to package assessment products that have been developed specifically for certain vertical markets. This is driven partly by the fact that vendors are finding that it makes sense to promote and sell products in a more targeted manner. Health care is one of the hottest examples of this trend, but I have also noticed increases in product creation and promotion in manufacturing and financial services as well.
6. More Products Designed for the Middle Market. The labor-intensive nature of assessment tools has not traditionally lent itself to use by smaller or mid-sized companies. An increasing number of vendors are creating turnkey products designed to overcome the need for professional services and large-scale data collection exercises. As interest in assessment continues to grow, these products will be well positioned to provide value for a wider range of companies.
7. Movement into Job-Search Process. An increasing number of job boards/career portals are discovering that adding assessment can provide much better search/match functions than do traditional methods, such as keyword search. I have been pushing this idea for the past five years, but this year will provide a noticeable increase in the use of assessment tools as part of the DNA of many job boards.

While the above represent what I feel to be very positive trends, I also see the perpetuation of a few trends that I am not so wild about. These things include:

8. Continued Fragmentation. Most users of assessment seem to still take a test-focused viewpoint in which the emphasis is on the use of one specific test rather than on the result of a coherent process. While some situations warrant this type of approach, it is best to build a process in which assessment data is tightly integrated into other decision-making data. The good news is that all of the trends outlined above represent movement in the direction of less fragmentation.
9. Tactical Focus. Assessment is still being used in a highly tactical manner. It is most often considered as a way to fight fires, as opposed to a preventative measure. As consumers experience more success with the use of assessment tools, a more strategic focus will result. It will be some time before we see this type of usage occur, but we are headed in the right direction.
10. Continued Lack of Proper Evaluation. My head hurts from repeatedly banging it against the wall over this issue. Yes, it is hard to create the buy-in needed to properly evaluate the impact of assessment tools. However, without proper evaluation, it is very difficult to clearly demonstrate ROI and build a solid business case for the use of assessment tools. While vendors are trying to help design products that can assist in this area, proper evaluation requires commitment from the consumer. The consumer is the one who needs to collect the data relating to the business impact of assessment. While this is not an easy task, if we can put a man on the moon, we should be able to collect relevant performance data.

This promises to be a great year for screening and assessment. Continued interest by consumers has created a buzz amongst vendors who are, in turn, creating products designed to eliminate the headaches often associated with using assessment. Of course, there are always tradeoffs. The more turnkey a product is, the less it will be optimized for a particular local situation. The degree of customization required is a business decision that should be made based on a thorough needs and cost/benefit analysis. Some things never change!

Want to help us learn more about what is going on with the use of screening and assessment? Please take five minutes to complete our survey. We will be sharing the results with the ERE community this spring.

Every so often, I come across shameful hiring information included in newsletters. I always thought journalists were supposed to research their facts; however, in a recent career newsletter, there were three articles that immediately got my attention. If any readers come across articles like these, may I suggest you flame the author for reporting pure nonsense to both recruiter and applicant.

Secret Documents

One article suggested the “secret career document” approach. According to the author, this is a powerful technique created by one of California’s top marketing professionals. This professional allegedly guarantees applicants will immediately be in the top of the “must-hire” list for any position they seek. According to the author, handing this document to the interviewer literally encourages him or her to hire on the spot! What a concept! Where can I get one?

I don’t know about you, but I immediately have questions when I read this kind of stuff. For example, is the author clueless or shameless? Do journalists believe jobs are so simple that anyone can qualify? Or, are they recommending an elaborate program involving kidnapping and extortion?

I am not sure what to make of this claim. It seem like the writer believes interviewers are so inept they can be easily tricked by reading secret prose. He also seems to believe that anyone can have any job he or she desires just by writing a great letter. (The real truth is that he is probably taking a break from his regular job as a barrister finding beneficiaries for a $10 million Nigerian slush fund.)

Yes, smooth letters and job summaries can be helpful when seeking a job, but let’s get real. This stuff is silly. It puts interviewers in a bad light, makes every applicant think intro letters outweigh skills, and ranks right up there with promoting radioactive mouthwash as a cure for gum disease.

Freelance authors: Be careful with what you report. Recruiters and interviewers: Be careful with what you write because someone might just believe it.

Rehearsal Programs

Rehearsing the interview was another forgettable suggestion. These were recommended when encountering interviewers armed with insightful questions like, “What would your friends say your greatest weakness is?” or “What kind of tree are you most like?” or “What is the sound of one hand waving ‘get lost’?”

One consultant suggested foolish questions were often company policy. (Sure. And so is feeding everyone free coffee and locking the restroom doors.) If senior executives have enough free time to make silly questions “company policy,” then you can be assured they are wasting good money hiring poor employees. Zen-like questions might be good for achieving inner peace or staffing a Buddhist monastery, but they tell you more about the interviewer than the applicant. In fact, a wise Zen master might even say these interviewers have achieved a state of no-mind. I could buy that.

Don’t get caught in the leading-question trap or think you can discover hidden knowledge by asking questions that have unverifiable answers or are intended to provide deep insights into an applicants’ character. Smart applicants will outsmart you every time.

Beyond asking a few general questions to start the conversation flowing, make the majority of your questions job-related, specific, and hard to fake. If an interviewer cannot figure out what these questions are, then it is a sure sign he or she will default to the Sigmund Freud approach (but without the education, experience, or skill).

So, if “favorite tree,” “describe yourself,” and other unverifiable questions are at the top of your interview list, sign up for a course in behavioral interviewing. It will probably only cover about one third of what you need to know (the other two thirds are job analysis and standardized scoring). But, at least you will make a better impression on applicants than pretending to be a dead, 19th century psychoanalyst.

Bad Test Practices

Some people would like you to believe that motivation or personality tests are all you need to predict success. But, aside from unskilled jobs, doesn’t every job require some form of hard skills? Job performance is a two-sided coin: it takes both job skills and job motivation. Unmotivated, highly-skilled people are considered underachievers. People with different personalities perform the same. And people with the same personalities perform differently. Highly motivated but unskilled people are “train-wrecks.”

Does motivation accurately predict everything you need to know about job skills? Only if the sole product you have to sell is a personality test.

Academics make a serious distinction between traits, styles, characteristics, motivations, and so forth. From a hiring perspective, though, all we generally want to know is a person’s “AIMS.” That is, does the applicant have the “Attitudes, Interests, and Motivations” associated with job performance?

Applicant motivation is one of the harder areas to evaluate. It is easy to fake; unemployed applicants are generally motivated to get hired and tend to say nice things about themselves. So, as a general rule, questions about motivation tend to lead answers. Smart folks know how to fake well. In essence, you can never be certain whether applicants who answer motivational questions are being truthful, don’t have a clue about job requirements, or just faked you out. That’s a problem.

In my experience, motivation is accurately measured using hard-to-fake written tests that directly relate to job characteristics. “Directly related” means test items are not generic. The DISC, for example, is a four-quadrant generic test. This is OK for understanding that people are different, but it’s insufficient for predicting job success. Why? Jobs are complex. Dr. John Holland, for example, identified six separate job motivations: investigative, artistic, realistic, enterprising, social, and conventional. Holland observed that certain job characteristics appealed to different types of people (job skills notwithstanding). Additional research by other investigators eventually showed conscientiousness, extraversion, and not being neurotic affected job performance. In summary, a thorough evaluation of job fit and job attitudes requires measuring roughly nine dimensions…not four.

“Hard to fake” is another matter. It means the test delivers consistent results and test scores directly related to job performance. The DISC (and its clones), for example, is an “ipsative” test. Applicants are asked to select which adjective is most like them and least like them. At the end of the test, the “mosts” and “leasts” are summed and the totals manipulated to get a D, I, S, and C score. Nice and neat, right? Sorry.

Are all the adjectives equally weighted? That is, if Jamie selected 6 of the 24 dominant adjectives and Johnny selected a different set of six dominant adjectives, are their scores the same? Nope.

But that’s not all. Johnny and Jamie probably compared their six “D” adjectives to entirely different sets of “I” adjectives, “S” adjectives, and “C” adjectives. So, even the comparisons were different. Are Johnny and Jamie still equal?

Finally, after a little adding and subtracting, the test goes out on a limb and predicts that one profile represents a public face, one a private face, and one a back-up face, all from comparing four sets of 24 words. Please! If personality was that simple, psychologists would have stopped arguing about it years ago. Someone’s preference for 100 words predicts just that: a preference for 100 different words.

In summary, while Johnny and Jamie appear to have similar scores, they described themselves using totally different standards, used an insufficient number of factors, made individualistic comparisons, assumed equal weights, may or may not have matched job standards, and were backed by an extensive narrative based on limited data. Four factors do not cover everything you need to know. Comparative (e.g., ipsative) scoring doesn’t allow you to compare a person to a job. The DISC is probably OK for training, but it does not meet professional standards for hiring. Does this seem like a trustworthy hiring approach to you?

Conclusion

If you think finding applicants is hard work, then try testing their skills in a field characterized by an abundance of bad science, poor advice, and misinformation. It’s enough to make an applicant believe in secret documents and rehearse his or her interview answers before taking the interviewer’s DISC profile.

Get used to it: unless your organization hires everyone who applies, you are testing. Some people (even attorneys who should know better) vigorously deny that their organizations test applicants (pssst?interviews are tests!).

Whether an organization uses verbal questions or written questions, they both have the same objective: to separate qualified applicants from unqualified ones before spending big bucks on salary, benefits, and potential lawsuits. Tests are tests.

Now?let’s discover whether your test is working for you.

A Good Test

Separating a good test starts with reliability. Suppose an applicant takes a test on Monday and on his way out, you deliver a carefully aimed blow to the head sufficient to cause short-term memory loss (but not permanent damage).

After he gets out of the hospital, you invite the applicant back to take the same test a second time (with the promise of safe passage). Will he score roughly the same? That is, can you trust the scores to remain consistent from one time to the next?

This is called “test-retest reliability.”

Reliability means you can trust a test to deliver similar scores regardless of when it was taken. Otherwise, you would never know whether it was accurate.

Interviews, for example, are notoriously unreliable. Interviewers tend to like or dislike applicants; they may ask different questions of different candidates; they may think the objective of the interview is to get to know the applicant (wrong answer!); they tend to rate applicants based on personal appearance; and sometimes interviewers just talk about themselves. Interview test-retest reliability is pretty low.

Reliability is not limited to interviews. It also applies to many popular tests used in training, especially ones that measure personality type. Type-tests are fine for workshops and communication classes, but even some of the most popular ones are filled with reliability problems. Independent reliability studies show scores from a popular four-letter type-test tend to change from one time to the next. So, test authors, which score is the “real” score? The score on Monday? Tuesday? Last month?

Before you subject any applicant to a test, examine the vendor’s manual carefully and search for a section on “reliability.” You want proof the vendor knew enough to study the reliability of:

• Each test item (item analysis).
• All test items (inter-item reliability).
• The first test half compared to the last half (split-half reliability).
• The same people at two different times (test re-test reliability).

If you cannot find any reliability data, then your favorite test scores probably change from day to day. The next time you buy a pound of cheese, wouldn’t it be nice to know you were really getting the weight you paid for?

Hopefully you see that unreliable tests are a dead end, especially since most organizations want their tests to predict performance.

Using Test Scores for Prediction

Predicting job performance means that a reliable test score is directly related to job performance. The word “directly” means two things. First, it measures something that affects job performance. Second, the scores correlate with ratings. A typing test, for example, is clearly linked to jobs that require keyboard skills. If your organization still has a typing pool, the scores probably indicate the amount of work a typist can do.

But are keyboard skills always linked to job performance for management? Should we fail candidates who could learn keyboard skills in a few weeks or months? Do we know if applicants are physically unable to operate a keyboard?

Accurate prediction is called “validation”?and if you thought reliability was complicated, you ain’t seen nothin’ yet! Validation requires knowing clearly what skills are necessary for the job, and doing sufficient analysis to show test scores are statistically correlated with job performance (i.e., the test content and job requirements are causally related).

Otherwise, you are predestined to turn away qualified people and hire unqualified ones. Is that wrong-headed or what?

Why Should I Care?

If your objective is finding and filling, then you probably don’t. Stop right here, get some coffee, and don’t send me any nasty-grams. I assure you reading this article will be a colossal waste of your time.

However, if cutting turnover in half, doubling individual productivity, reducing training expenses, and building a solid base on future-qualified employees is attractive, then you need to know this. These claims are all normal for an organization that uses reliable and valid tests. Why? Their tests screen-out unqualified applicants. In case you are wondering, only about one applicant in six (on average) can pass a series of validated tests. Put another way, only about one applicant in six can demonstrate skills required for the job.

Ever hear about the 80/20 rule?the one where 20% of the people produce 80% of the results? It’s amazingly close to a one-in-six hiring ratio. Think about it. So if you care about making the biggest splash ever in the company pool, then continue reading.

A Bad Test

A bad test is one that an organization uses consistently, is backed by folklore and plenty of personal anecdotes, but has never been critically evaluated. Bad tests usually come out of corporate training programs. That is, a workshop participant who answered 10 questions about being a thorough planner was “amazed” when the test reported he or she was exceptionally organized. Next step?.use it for hiring!

Folks, personal agreement with test scores is not a reliable and validated way of predicting job performance. It is only a summary of how someone describes himself or herself. It is a self-reported description. Is the person actually as organized as he/she says? Or are they faking? If they are not faking, is organization important to job performance?

Defining the Job

This is a tricky area. The secret is to define the critical skills that directly affect job performance. This might include learning ability, problem-solving skills, persuasiveness, and so forth. The key to defining job requirements is to identify behaviors leading to job success or failure. It sounds weird, but you don’t look for results, just the behaviors that lead to the results.

If you cannot clearly define the key job skills, then there is nothing to test. The 1978 Uniform Guidelines suggest job competencies be based on job requirements and business necessity. I don’t know about you, but that sounds pretty good to me. Amazing! The government recommends organizations test for job requirements and business necessary. If anyone out there can suggest something better than basing a test on job requirements and business necessity, I’d like to hear it.

To reiterate, your test first has to be reliable. Then you must know what to explicitly measure. To make sure the test works, determine whether test scores predict job performance. We call this step “validation.”

Throw It On the Wall and See What Sticks Approach

Here is a sure clue to wrong-headed hiring practices. It goes like this. A vendor has a general personality-style test (we’ll make a fanciful assumption that it passes professional reliability standards). The vendor herds high producers into one group and gives them the test. He examines the averages and declares, “Yea, verily, these scores doth become our target!” (vendors like to use old English?it sounds so classy!).

Whoa?not so fast.

How does one define high-producer? By results or by actions that lead to results? It makes a big difference. Individuals in the high-producer group could have used different skills to get there. Some might be good politicians. Some might be very smart. Some might be taking credit for others’ work.

What about the confusion between correlation and causation? Just because ice-cream sales and shark attacks are correlated does not mean that one causes the other. Almost anything can be correlated, but not everything is causal. If you sort through enough garbage, you are likely to find correlations between cookie wrappers and hotdogs. So what? Your goal is to find a correlation between hotdogs and hotdog buns.

The “see what sticks” approach has a few natty problems. Sure, it looks scientific, but what good are decisions based on wrong-headed performance criteria, wrong-headed clustering techniques, and wrong-headed statistical analysis?

Job-Match Approach

The job-match approach is scientifically similar to the “see what sticks” approach, except worse. Some types of tests say certain occupations have similar styles: Introvert Sensing Thinking Judging (ISTJ) for example.

Before you use this stereotype for hiring, ask yourself if all the people in the same occupation do the same thing, or do they all perform equally well? Did their personality style cause them to be an engineer? Are these folks extreme ISTJs or are they marginal ISTJs? Do their organizations all have the same objectives for the job?

Conclusion

Everything starts with the human elements of job requirements and business necessity. Human elements are seldom included in job descriptions or job evaluations. You have to dig for them. If you cannot test/interview for specific human elements, your tests will probably be inaccurate.

All selection tests have to pass rigid standards for reliability and validity. Reliability means the test delivers consistent results time, after time, after time. Validity means the test scores accurately predict job performance and should be done carefully.

It is a grave mistake to assume any group of performers has equal skills. For example, some salespeople are great repeat sellers, some are great cold callers, and others are great service people.

They all might be high performers but for entirely different reasons. It is a big mistake to assume characteristics or traits correlated with performance actually cause performance.

No facet of assessment has experienced as much change over the past decade as has the reporting of candidates’ results. Those of us who worked with assessment back in the dark ages can attest to the frustration experienced with the reporting of assessment results.

In those days the following were the norm:

• One would simply overlay a scoring key, enter in numbers into a worksheet, make some calculations, and arrive at a score for each applicant and perhaps a percentile rank based on an ancient set of norms.
• In some cases one could fax off an answer sheet and receive back a narrative report summarizing an individual’s performance.
• Examining aggregate data for more than one applicant required creating an Excel file or similar table by hand. These often had embedded formulas to help provide the information needed to make decisions about an applicant relative to other applicants and/or some set of job standards.

The commonality in the above activities was that no matter how it was done,