High-Level Observations

• The rise of talent management. This seems to be the age of  “talent management” when it comes to the use of technology in HR. I saw a ton of companies offering “talent management systems.” These platforms use technology to cover a broad footprint of key HR areas/functions such as branding, recruiting, onboarding, learning, development, and communication. Talent management products are starting to provide HR practitioners with a technology based backbone that will allow them to integrate major HR functions. The integration of more functions into one platform is a trend that can have significant value given the traditional walls that tend to exist between the major areas of HR in larger organizations. There seems to be variation in the functions offered by the various talent management platforms as well as some gray area around what defines a talent management product. These gray areas are nothing new, and are indicative of the nature of an industry-wide trend that is causing vendors to jump on the bandwagon. While my overall thoughts about talent management products are definitely positive, one wonders how many vendors have just dubbed themselves as “talent management” providers to be trendy, and have not significantly changed their products.

• ATS Not Hip Anymore. Is it just me, or does there seem to be a significant decrease in companies selling products referred to by the term “ATS”? I could not help but think that much of the momentum in the use of hiring and technology seems to be in the idea of the broader, strategic, idea of talent management and less in the more tactical area of applicant tracking. The concept behind applicant tracking and its related functionalities are still of great importance; it just seems that the term itself is losing favor as traditional ATS functionalities are being baked into other products such as advanced screening/assessment management platforms and talent management systems.

• Goodbye paper resume. It seems that there is continued movement toward removing the resume from the hiring process. This is definitely true of the paper resume, but there is a broader movement afoot to deconstruct the resume and extract the types of data it usually provides (i.e., skills and experience). Key information about candidates is being collected via other opportunities in the search and application process such as the creation of detailed profiles that have fields that guide the entry of important candidate information in a standardized format. Parsing, another method of deconstructing resumes, seems to be continuing to evolve as a means of extracting important data from resumes. Resume deconstruction methods allow for much more efficient searching, and matching between candidate data and important job requirements. These methods are part of an overall trend that will see the integration of a variety of candidate information into a digital platform that will allow it to be standardized, categorized, and compared to key requirements for a job or career. While we have a long way to go toward the complete death of the resume, technology is helping us to continue the slow march towards the inevitable.

• Web 2.0 is hot. It almost goes without saying that the latest in HR technology is leveraging the benefits of easy access to information, communities, and data that are at the core of Web 2.0. Web 2.0 was everywhere at this show. It seemed to me that the applications of web 2.0 seemed to be concentrated more in the space of performance management products, but there is no doubt that it is having a large impact on the thinking of those creating recruitment-oriented products as well. I saw a heavy focus on the use of social networking tools such as Twitter and Facebook and on products that provide companies with platforms on which to build communities. The data-driven nature of web 2.0 provides a good deal of promise for the area of recruiting and hiring.

• We’ve come a long way. The products I saw at this show provide a sign of continued progress and evolution that seems significantly more advanced than the products available five years ago in many ways. While the core function of many products may remain the same, we are continuing to develop new ways to connect people and information. The products I looked at seem slicker, cleaner, and more usable than ever before. While it is often hard to cut through the smokescreens when one is looking at products on the tradeshow floor, the overall level of tech-savvy seems higher than ever. I was able to see firsthand the positive impact of technology on a variety of recruiting- and hiring-related products. Of course keeping up and separating faddish technologies from those that truly are game-changers will continue to be a challenge in years to come.

Assessment-related Trends

• Assessment is still an outsider. While there were a decent number of assessment vendors at the show, as an overall area of HR technology, assessment is still a very small piece real estate. This makes sense to me given the problems that organizations seem to have in understanding the value assessment can provide.

• Two directions of evolution for assessment. Technology is helping assessment to slowly evolve in two directions. The first of these involves assessment being integrated into the functions provided by bigger, broader systems and products such as job boards, ATS, and talent management. Embedding assessment into other products makes sense when one understands that the core value of assessment is to provide information that can be used in conjunction with other information to support informed decision-making. Pre-employment assessment actually may have the most value when it is an integrated part of a process-based approach into which its results are integrated. This trend has been very slow to develop, but we will continue to see a trend toward embedded assessments. The second direction in the evolution of assessment is toward the productization of assessments to be sold transactionally. While assessment has been sold transactionally for decades, the present state of evolution leverages millions of data points to help provide a new level of clarity about what content predicts certain traits, behaviors, and outcomes. This evolution allows for off-the-shelf products that are more accurate than ever before, while providing documentation to support the relevance of these products for specific jobs.

• Hello, talent management providers? Assessment seemed to be conspicuously absent from the slate of services provided by most talent management systems. While assessment is part of the deal for some of the big players in talent management, it does not seem to be a core part of the concept at this point. This is upsetting given the value assessment can have, not only during the hiring process, but throughout the entire employee life cycle. Anyone who considers themselves to be a player in the realm of talent management should strongly consider embedding quality assessment tools.

• Platforms evolve nicely. The software platforms that accompany pre-employment assessment products are evolving nicely. I refer to these as “candidate management systems.” Today’s candidate management systems offer many of the functionalities once seen only in ATS products. This evolution should help support the value proposition of assessment as products become easier to use and provide additional decision-making support.

• Simulations still lacking. While there has been some definite positive movement toward the creation of more advanced simulations, we still need to be pushing ahead in this area. While there are some nice simulation products currently available, the future of hiring will continue to move toward the increased use of simulations. Not enough steps are being taken toward the development of truly innovative and unique simulations. This is an artifact of a lack of buyer interest and the absence of “killer apps” that are needed to allow simulations technology to advance.

When it comes to HR technologies that support the recruiting and hiring process, we are moving in the right direction and have come a long way over the past five years. The technology level available for assessment both as a core process and as a supporting cast member is more advanced than ever. However, there is a lot more that we can be doing to integrate assessment into the products that seem to be marking the trends in HR and technology. Assessment still seems to remain the misunderstood stepchild of HR, which is a shame since there is so much evidence for the value it can have. I remain optimistic that assessment will continue to be brought more tightly into the fold in years ahead.

At ERE’s Fall Expo in Hollywood Beach, Florida, recruiters will meet the newest businesses to launch. Two of the founders will talk about how their respective companies are harnessing the power of video to help recruiters make better hiring choices and save the environment while also saving the hiring company a few dollars.

Greg Rokos, founder of FutureResume.com and its affiliate, GreenJobInterview.com, and Darryn Severyn, founder and CEO of Interactive Applicant, will try to convince recruiters that video resumes and video interviews are effective ways to screen candidates without the cost or carbon expenditure of bringing them onsite.

FutureResume.com is where candidates post a video and standard resume, the latter searchable by the usual means. Then, instead of bringing in a candidate for a first meeting, they can be interviewed online.

Interactive Applicant takes a little different tack, pre-screening applicants via an automated series of questions that candidates can be required to answer via video, audio, text, or any combination. Then the recruiter can review the candidate’s presentation skills before bringing them in.

SnapTalent’s CEO and founder Sumon Sadhu will describe how his company’s online advertising service helps recruiters and hiring managers source better candidates. It’s similar to a keyword marketing campaign but places targeted ads on content sites, rather than on search results pages.

Jeff Stewart, Urgent Career’s serial entrepreneur founder, will show how linguistic technology can be used to match sales candidates to jobs. That’s different than voice analysis, though there are some similarities. Just how it works and how effective it really is are questions that Stewart will be answering next week.

If the Spring show is any guide, these founders will get questions as tough — maybe tougher, since the audience knows recruiting — as any venture capitalist will ask. Hardly a shy bunch, the Spring ERE audience point-blank asked that crop of company founders and executives how they intended to make money and why an employer should do business with them.

Ben Yoskovitz, founder of Standout Jobs and one of the presenters at the first Startup Forum, told us that since the show he’s learned more about the HR industry than he thought possible. “The panel was a good place for us to start getting feedback, ” he told us recently.

His company provides easy-to-use software for smaller companies to build their own career sites.

Since the spring, Standout Jobs has grown to over 200 customers. While still a free service, that will soon change. An upgrade to the service is also planned. And, Yoskovitz says, now that the intensive testing and learning period is mostly behind the company, promotion of Standout Jobs is the next major effort.

Path 101, a content-intensive community networking site, is still in alpha, which, though growing in depth, is about where it was last Spring. But then, it’s an ambitious effort, which founder Charlie O’Donnell, at the forum, described as a site for job-seekers to research their career options before they apply for jobs. The site itself says it is a place where “Job candidates can figure out what “people like me” are doing with their careers and the site aims to be the first stop for career research.”

At Jobscore, CEO and founder Dan Arkind told us that the last six months have been a learning and testing period. “Not much to report,” he said at first. After a little prodding he said the resume-sharing site has been making inroads into the smaller employers the company is targeting. Especially those in the San Francisco Bay Area, where Jobscore is headquartered.

The company was developed to help smaller businesses source better candidates by sharing resumes and easily post jobs to one or multiple sites. Companies can choose to pay to gain access to the resumes or earn free access by sharing resumes. So far, Arkind said, 96 percent of the customers share.

He has intentionally kept Jobscore low-key. Soon, he said, it will be making a bigger splash. When? “When it’s ready,” Arkind said.

Finally, there’s VisualCV, a site where jobseekers build an online presentation of their experience, background, skills, and more using text, multimedia, and even work samples. It’s an adjunct to the standard resume, not a replacement, at least not yet. Though COO Doug Meadows told us, “What we want to do everyday is wake up and replace the resume.”

We didn’t get to talk to our presenter co-founder Clint Heiden, but Meadows said the company has been “going gangbusters.” VisualCV has been the most visible of all our startups. Cheezhead, alone, has featured the company no less than four times since the Startup Forum in early April. It has a new CEO.  It’s also added new features, most recently a VisualCV Marketplace.

There are now 800 companies signed up with VisualCV, meaning they accepting the VisualCV and have their own posted on the site. Participation in the program is still free; the company is generating revenue from private labelling VisualCV to business groups, associations, alumni organizations and others. The China Business Network uses it to help its thousands of members better connect. Search firm Heidrick and Struggles uses it for its elite group of candidates. A few firms are also using the site to search for candidates, Meadows said, paying a findersfee when a VisualCV member is hired.

What will the next six months bring for these startups and for the four new companies presenting next week? That’s an even tougher question today than it was last Spring because of the economic conditions in the U.S. and around the world. We don’t know how they will adjust, but you can be sure that’s a question our latest crop of presenters will be asked.

Many companies are not prepared to go through the offer step of the process. As a result, they damage the relationship with the candidate. This leads to one of two unfortunate conclusions. Either they lose the candidate or the candidate comes on board, but with scar tissue. Applying some of the best practices from the sales world into a sales talent screening program helps to avoid that scenario.

The offer stage of the hiring process parallels the proposal phase of sales. Best practices in sales say that you don’t present a proposal until a thorough needs analysis has been completed. If a sales person is presenting a proposal to a prospect, he has acquired the information needed to design a solution, has discussed budget, has a full understanding of their solution requirements, and has set an expectation on pricing. This is certainly the case if the salesperson is going to be successful in winning the account.

Looking at this process in relation to the offer stage of the sales talent screening program, many of the same best practices from sales hold true. During the screening program, information needs to be gathered from the candidate to determine their financial requirements. Unfortunately, many sales talent screening programs focus exclusively on screening the candidate for fit, but do not consider the needs for the offer phase of the process. This leads to a last-minute scurry to mine the information from the candidate, or they design the offer blindly. Neither of those are best practices for the offer stage.

In sales, it is said that if you are going to lose, lose early. This prevents you from making a huge investment in a relationship that will not generate revenue. The parallel to screening sales talent is understanding the financial requirements of the candidate early enough to stop the process before over-investing in the relationship. There is no point in continuing a process with a candidate who requires a compensation level 25% above what you can offer. This probably seems logical, but hiring executives rarely focus on this as a de-selection element early in the process.

Just like discussing pricing with a prospect, the financial-needs discussion requires finesse. The candidate knows that you are asking questions about their financials, just like a prospect knows a sales person is fishing for budget information. The better-skilled salespeople tell their prospects, “I don’t want to waste your time by getting you excited about a solution that will not fit in your budget constraints…”

In much the same way, this discussion can be had with the candidate, “I don’t want to excite you about an opportunity that might not be a match for your financial needs. As you look at making a change in position, what thoughts have you given to your compensation requirements?”

With continued finesse, you can dig further into the mix of salary versus commission. Some candidates may rebuff this discussion as they feel the information will be used against them. In some instances, they are justified for having that concern. Hopefully, that is not the case in your company. We’ll come back to this point later. The bottom line is that the two goals of this phase are to gather information that allow you to formulate an offer and to de-select those candidates whose requirements exceed your financial package.

In sales, the proposal phase should not be like a magic show. The prospect should not be shocked by what is included in the proposal. In essence, the proposal is the documentation of what has already been discussed. No surprises. The same holds true for candidates. The time to review the compensation plan details is not after they are hired, or even at the offer stage. The compensation plan should be reviewed at the point where you have a genuine interest in pursuing the candidate and they have a complete enough understanding of the company that they will be able to comprehend the compensation plan.

One of the core requirements associated with any process is that it is measurable. The offer phase of the sales talent screening program should be measured statistically to determine effectiveness. The key statistic is number of offers made versus ones that are accepted. If the acceptance level is less than 80%, the process should be reviewed by asking the following questions.

1. At what point of the process are the candidate’s financial requirements reviewed?
2. When it is known that the candidate’s financial requirements exceed the package, is the candidate removed from the process?
3. At what step is the compensation plan reviewed with the candidate?
4. In what level of detail is the compensation plan reviewed with the candidate?
5. How often is the initial offer to the candidate rejected, and subsequently, negotiated successfully?

The last question in the list above ties back to my opening position about damaging the relationship. Again, this ties back to lessons that can be learned from sales. Many years ago, a procurement training specialist shared a pearl about the counsel he gives to salespeople who ask about pricing strategy. He said, “Provide us with the best pricing that you feel comfortable providing and either way you are happy.” This always puzzled salespeople so he explained further. “If you provide your best pricing and are selected, you are happy because you won the account. If you are not selected because we found lower pricing elsewhere, you are happy because you would not have been happy at that price point. Again, either way you are happy.”

Consider this when making an offer to the sales candidate. Develop an offer based on what was learned from the candidate that represents the best offer you are willing to make. Early in the process, tell the candidate that you don’t negotiate offers, but rather put your best offer on the table upfront. It demonstrates a professional message to the candidate and reduces their fear of attempts to lowball them. When companies negotiate offers, while they may “win” the candidate, they damage the relationship. This person is onboarded with the worst scar tissue of all, a lack of trust. The salesperson will always be on the lookout for the company to try to cheat them.

As with any component of the sales talent screening process, preparation is the key to success. Organize your team and design a process that achieves your desired results. This will allow you to create longlasting, fruitful sales marriages.

Like white? Then interior decorating is for you. (Too easy. Everyone knows white goes with everything.)

How about if your favorite colors happen to be black and red and orange? Maybe you just really like Halloween. Otherwise, you are “The Evaluator,” says a press release from CareerBuilder (profile; site), which just added a color wheel (parked on the old CareerPath.com website) to help jobseekers better assess their personality.

Before we get scolded for making light of a serious assessment tool let us note that the Color Career Counselor has been scientifically vetted with the results published in the North American Journal of Psychology. You can read the paper here, but fair warning: it’s full of the kind of statistical analysis we avoided in college.

The CareerBuilder service is powered by the Dewey Color System, which uses color selection to determine your personality type and traits. These then suggest certain types of careers that others with smilar personalties have found rewarding and successful or which typically attract people like you.

Dewey offers its colors-based assessment to companies as a pre-screening tool, touting the Dewey Employee Predictor as “the world’s first nonlanguage-based test that defines occupational interest, workplace behavior patterns, styles, traits, and temperaments, as well as an additional 16 personality factors.”

Invented by Dewey Sadka, a veteran recruiter who founded and ran his own employment agency, Temp Force, he’s written several books on the use of color in personality assessment and employment. Sadka’s latest book is on using colors to find love. His clients include Georgia-Pacific, Honeywell and SHRM. And now, CareerBuilder.

Taking the test takes only a few minutes. And we will say this, the analysis was right in more ways than we would have thought. It helps if your monitor is properly calibrated. We couldn’t see much difference between indigo and blue.

Now about that black and orange and red thing. According to the CareerBuilder press release, those who picked those colors seek “the most efficient way to accomplish tasks. Using facts, compiled from past mistakes and successes, you deliver strong opinions with valuable perspectives that maximize the bottom-line.”

And the favorite color for that bottom-line presumably would be black.

In this article, I praised the progress being made in understanding the value of quality assessment products in the modern hiring process.

I strongly believe that the words I wrote in 2006 are as relevant today as they ever were; in fact, they are more relevant now than ever. The days of the big job boards and their keyword matching strategies are numbered.

A lot has happened in the world of online recruitment in the last decade. The focus of most of the evolution has been on candidate quality and on increasing accuracy. This is in opposition to the focus a decade ago, which seemed to be on quickly delivering a large volume of candidates. What we have learned since then is that without a way to quickly evaluate candidate quality, we end up saddled with information overload. So, the objective for recruiting products is shifting to one that is focused on the alignment of personal characteristics with those that are required for performance on the job. This is a good thing. This movement has definitely been aided by the choice of several leading career portals to embed candidate quality into their bag of tricks.

A slice of the content of the original article from 2006 appears below. It has been updated to include a few examples of a few career portals that are now using assessment to help them better serve their clients (both job seekers and employers) by providing them with a way to quickly focus on candidates who have what it takes. Before we get started, I want to offer a few disclaimers here.

1. I am not saying that using assessment as part of the job search and match process provides all the data needed to make a good hiring decision. Rather, I am saying that it provides an excellent way to address the idea of “garbage in, garbage out.” By creating an applicant pool of individuals who are at least qualified and motivated, we are taking a very important first step towards ensuring systematic quality in the hiring process. I think of it as pre-screening that comes before pre-screening because it provides an applicant pool that will give one much better odds of making a successful hire.
2. In this article I am going against my policy of not ever naming the names of any vendors who provide the type of products discussed. Please note that none of the vendors discussed here have been included due to anything other than my personal opinion that they are good examples of the trend I am documenting. I was not compensated by them, nor does my inclusion of them in this article mean they are the perfect solution for your problems.
3. Although I am an expert in this space, I do not know everything, and there is definitely a chance that I failed to mention a company that provides the exact type of service I have described. I encourage someone from such companies (or a fan of one of these companies) to contact me to let me know what you are doing.

Below are some relevant highlights from my original article.

Folks are finally starting to get the idea that assessments are but one component of a broader process designed to help hiring professionals make systematic predictions that result in good hiring decisions. There’s much to be gained from this process-oriented approach, so it has been great to see assessments being used to help at various phases of the hiring process.

Despite the continued integration of assessment, there’s one area in which assessment has been underused. This area is the use of assessment tools during the job-searching process. Specifically, the integration of assessment tools into the searching/matching component of career portals.

Traditional methods available to users of career portals for locating jobs are extremely crude, consisting mostly of keyword searches or matching based on simplistic profile elements.

The fact that this basic process has endured as the standard for almost a decade now clearly reflects the quantity-over-quality focus that has reflected the marketing-centric attitude of many job boards since day one.

Blocking Out the Noise

This crude matching process results in excess noise. For applicants, noise means they’re presented with, and encouraged to apply for, a ton of jobs for which they’re not qualified. For hiring professionals, noise means no mechanism to assist them in making an initial high-level determination of applicant quality.

Noise makes it difficult to make good hiring decisions because it greatly increases the chance to make systematic errors.

The good news is that I am starting to see a shift in focus that will go a long way toward a reduction in noise and an increase in the ability to differentiate candidates based on various quality factors.

This shift involves the use of assessment tools as an integral part of the services provided by career portals. The use of assessment to help fill the hiring funnel with applicants who have the attributes required for success has numerous advantages, all of which are explained by simple probability theory.

Think about it: the more qualified the individuals in your applicant pool, the better chance you have of hiring someone who has what it takes. Over the past few years, I have learned of an increasing number of companies who are seeking to change things by integrating assessment into the matching process.

The basic aspects of this include the following:

• Seekers create profiles. This part of the matching process works as part of a registration process, before a job seeker has even expressed interest in a specific position. Along with other information collected during this process, the job seeker creates a profile based on a short assessment of basic qualities such as work attitudes, personality measures, etc.
• Corporate users define high-level requirements. This part of the matching process requires hiring personnel to create a template of the basic things it takes to do the job well. This often includes standard things such as skills and experience but also includes qualities often measured by assessments (personality factors, dimensions of fit, work values, etc).
• Noise is squelched. When a job seeker searches for a job using the profile he/she has created, results include jobs for which they are suited based on the compatibility between their profiles and those of available jobs.

There are many twists on this formula but the basic idea is the same: deliver an applicant pool that has less noise and makes the recruiter’s job easier. This is the start to ensuring quality hires because it’s sure hard to hire superstars if they aren’t even in your applicant pool to begin with.

Admittedly, my description of this process is overly simplistic. In job matching, just as with making hiring decisions, assessment data should serve as one of many data points that are used to identify the best candidate for the job. Truly effective matching systems will use parameters that include other key determinants of success such as experience, knowledge, and skills.

Weighing the Pros and Cons

Overall, the use of assessment in the matching process offers the following five advantages:

• Reduces noise. Allows filtering based on actual job-related parameters.
• Helps determine corporate fit. This process is a good way to find and hire applicants who fit a company culture.
• Increases quality. Allows insight into applicant’s ability to deliver what it takes for success.
• Educates job seekers. Allows applicants some insight about themselves and how suited they are for a particular job or career.
• Results in branding. It can provide a good applicant experience, leading to support for employment brands.

Of course, there are potential drawbacks to the use of assessment as part of the matching process. These include the following five issues:

• Learning curve. Users must learn to understand how to define jobs in terms of relative importance of the parameters used for matching.
• Accuracy. It is critical to match the templates against which applicants are matched to accurate reflections of the job.
• Uncertainty. Predicting performance is a difficult task, to say the least. Assessment-assisted matching is not a panacea and does not eliminate possibility that mistakes will happen.
• Quality control. To be effective, assessments must be created using the proper methodology; substandard assessment products will produce substandard results.
• Legality. While the use of assessments in matching is not illegal, it does require awareness of some potential legal ramifications (see original article for full coverage of legal issues related to the type of products discussed in this article).

So who is actually using some sort of assessment as part of a career portal or job board? I believe the companies below are all doing great things to move us forward toward an era of embedded assessments that can help reduce the noise associated with finding and applying for jobs online.

Climber: Climber.com uses some really cool technology to create a comprehensive applicant profile that includes an assessment of work values. The complete applicant profile is then compared to a profile created by employers making job postings. Part of this process includes an employer work-values profile that helps capture the values of the organization in order to help ensure that users (both applicants and employers) are matched based on a mutual “fit.” Climber goes beyond the norm for this type of exercise by using sophisticated data analysis techniques and AI in order to help optimize the search/match process for its members. It is truly on the cutting edge when it comes to delivering candidate quality via tight searching/matching through the use of sophisticated data analysis techniques and AI.

Jobfox: Provides applicants with the opportunity to take a short personality assessment once they have registered. While the results of this assessment are not currently used to help match job seekers and employers, Jobfox members are able to see the results of their assessment and use them for self-understanding and developmental purposes. Jobfox also has a very nice skills matching technology in which applicants take the time to create a skills profile that is then matched against the skills profile that is created for each position that is posted by organizations using Jobfox. The result is a match score that provides candidates with an idea of which of the jobs available on Jobfox are a good fit for their skills and experience.

Monster: Monster has partnered with DDI, a leading assessment firm, to provide its clients with the ability to add assessments to any job posting. This product is known as the Monster Performance Assessment or MPA. The ability to provide the MPA for most job postings is made possible by DDI’s extensive experience with assessment tools as well as the data it has collected over the past several decades. The MPA package represents a groundbreaking offering, as it is the first time assessments have been sold transactionally as part of the process of posting job openings. The MPA does not use assessment to help candidates identify jobs for which they are matched; rather, it provides recruiters with much richer data about a candidate’s qualifications for the position of interest. This allows recruiters to quickly shortlist candidates based on a set of standardized, job related data. While there are still some details being ironed out, Monster’s approach represents an interesting and potentially effective way to help deliver quality. Monster is also the only one of the mega-boards that has been bold enough to step out with an assessment-related solution.

eBullpen: eBullpen is a job board that uses a personality assessment as part of the process used to match job seekers with available openings. eBullpen has been working on its process for several years now and provides a legally sound, useful approach. This approach asks companies posting jobs to outline the personality traits, activities, and requirements for the job. Job seekers provide data on their personality traits, work preferences, and qualifications. Matching alogrythms are then used to help match job seekers with available openings.

Each of these four companies are stepping out into uncharted territory and is doing so with a slightly different approach. The specifics of these approaches aside, the big picture is what matters the most here.

This big picture is the idea that each of these folks provide a way for job posters to break down a job into a series of key data points which are then matched to a series of related data points in a profile or assessment snapshot created by job seekers.

Detractors have often lobbied against this kind of matching, saying that no candidate (or company for that matter) is interested in taking the time required to create the profiles needed to fuel this kind of process. I firmly disagree. Good hiring takes an investment on the part of both the job seeker and the employer, and good investment often takes a bit of effort. The old adage, “you get what you paid for” along with my personal favorite “garbage in, garbage out” both apply to this situation. I am confident that as we travel further down the road of embedded assessment, the data will bear out the fact that the small amount of time taken to fuel the matching process will prove to be time well spent

Despite this, within the next five years I expect to see career portals offering an increasing amount of value by beginning to change the focus from quantity to quality, with assessments as one of the core elements of this switch.

I think all stakeholders involved in the hiring process, including applicants, stand to benefit from this trend.

Ravi Subramanian recommends hiring manager surveys and assessments. Michael Chernesky asks a very relevant question…who is accountable? Bonita Martin says recruiters and Steven Yeong says hiring managers. Joshua Letourneau offered some great insight based on his experiences. While many of his clients do measure quality of hire, their process often fails because of one of three problems. They don’t measure quality of hire beyond the first day; the hiring manager survey turns into a game since “recruiters are incentivized to make the survey look good”; or feuds spark between HR and the hiring manager. Josh’s solution to overcome these challenges lies in performance assessments that measure expected versus actual performance. Anyone out there having great success measuring QOH? Share your story; we would love to hear it.

Wednesday’s Question of the Day
Anna Patterson’s latest Internet search engine, Cuil, is backed by $33 million in venture capital and expected to outshine Google. I read negative reviews of Cuil and wanted to know if anyone is using it and seeing results. Glenn Gutmacher, Sourcer extraordinaire, is not a fan…it is slow and produces poor quality results. Without launching an alpha or beta version, Glenn believes they “set high expectations and they ‘way’ underdelivered.” T Tallis agrees. Amanda Blazo is more optimistic. She acknowledges Cuil’s shortcomings but encourages us to give it time.

Phone Screening Candidates
Diane Propsner wonders: before a third-party recruiter sends a resume to a corporate recruiter, How much time should he or she spend on phone screening and what questions should be asked? She gave us an overview of her process and the questions included. Andrew Stone agrees with Diane that 10-15 minutes is not enough time for building a relationship with a candidate. Tania Murray recommends “tailoring your cover sheet to your client to focus on just what they (candidate) told you is important to them.” According to Tania, it is critical to know the candidates goals and motivations. Joy Naui offers a different approach by suggesting that third-party recruiters first contact the hiring manager. The phone screening should include key points based on that conversation. Questions should include more than technical skill questions. If a candidate recognizes that you are genuine about finding them the right job, they will agree to answer more questions.

Thursday’s Question of the Week
After hearing Dr. Wendell Williams’ Selection and Assessment webinar on July 23, an attendee wanted to know if behavioral or situational screening questions are better and why. I thought I would ask you. Joe Payne is in favor of behavioral questions since they are a good indicator of past actions and can allow interviewers to dig deeper. Matt Cooper brings up an interesting point that most candidates are too prepared for behavioral questions with the amount of information available on the Internet including examples and preparation tools. Interviewers need to be creative and get more specific. KT Connor agrees with Matt and also cautions against self-reporting assessments instead recommending objective decision tools.

Are Job Boards Becoming Obsolete?
This is a hot topic of debate, check it out!

New I-9 Form Released…

U.S. Citizen and Immigration Services just released its new version of the I-9 employment verification form, so make sure to update your records. (You can download it here; note that the new expiration date in the right-hand corner reflects 6/30/09.) You can move to an e-file for these forms, and perhaps you should: employeescreenIQ says its data shows that more than 85% of paper I-9 forms are filled out incorrectly. And electronically verifying this step is certainly a “greener” thing to do, and companies like Verified Person, Inc. agree. Its CEO, Jim Davis, says his Verified Person I-9 solution “affirms Verified Person’s belief in promoting an HR process that benefits the environment.”

From Resume Fluffing to Conviction Bluffing…

The folks at employeescreenIQ also say one of the hottest background-screening trends centers around the importance of thorough background checks in a shrinking job market. In fact, considering the state of the economy, “the job market is destined to become even more competitive, which in turn could lead some individuals to stretch the truth in order to secure employment,” according to the company’s new list of 10 background screening trends. Also, employeescreenIQ says conviction rates among job applicants are on the rise, and points to a 56% discrepancy rate between what is reported on a resume and what is found when conducting employment and education verifications.

TMI on MySpace?

Does the idea of lurking on someone’s MySpace page make you yell, “Too much information!” or do you view it as a (lurid) source of quality screening? What about reading someone’s biographical data on LinkedIn? Though stylistically very different (LinkedIn is more button-down; MySpace is more, er, unbuttoned?), these websites provide glimpses into the backgrounds of hopeful job candidates. Just remember two things if you decide to spend more time on these sites: failure-to-hire lawsuits can be a real headache, be it from MySpace, Google, or any other website, and avoid “friending” Natasha Kone on LinkedIn at all costs.

Genie on a Job Board…

In the June 26 issue of WEDDLE’s Newsletter, by industry analyst Peter Weddle, he advises that the key to using the Web for employers and job seekers is to manage your expectations.

“What’s out of whack is our perception of what the Internet can and cannot do,” he says. Though, “this technology is probably the single most effective means of connecting” employees and employers, it takes time to work. He advises job-seekers, in particular, that “The Internet is not some genie,” nor is it “a magic carpet that can carry you off to employment bliss the minute you hop on. No, the Web for all of its reach and technological power is bound by the pace of the humans who use it at the other end–the employers and recruiters who turn to the Web to find new talent.”

Government Security Clearance…

Weddle also reports that ClearanceJobs.com’s recently released snapshot of security clearance salaries finds that those who have a security clearance earn 25% more than those who do not. This is not without hurdles, because “you have to be able to pass the intensive screening that’s involved in applying for a clearance, plus be in a job where a clearance is required for job performance. Even then, it may take a long time to acquire this advantage, as the backlog of Federal background investigations is already huge and growing,” says Weddle.

]]> http://www.ere.net/2008/06/26/the-new-i-9-form-and-other-screening-trends/feed/ http://www.ere.net/2008/06/19/how-to-hire-for-quality/ http://www.ere.net/2008/06/19/how-to-hire-for-quality/#comments Thu, 19 Jun 2008 09:15:34 +0000 Kevin Wheeler http://www.ere.net/?p=3218 Hiring people is rarely based on objective evidence and is, perhaps, the least-objective activity that organizations participate in.

When we see a candidate who meets a large number of our pre-existing conditions for employment (i.e., a candidate who has gone to a school our hiring manager likes; has worked at a couple of well-respected companies; or has written the right key words on his resume), we have already hired him in our minds.

Interviews are examples of how easy it is to abandon the tools of objectivity, the scientific method, logic, and the rules of evidence, for our “gut” or for “chemistry.”

While there is considerable evidence showing that testing candidates is far more likely to predict successful performance, we still rely almost exclusively on interviews. Though numerous researchers have pointed out the need to gather a variety of data about a candidate, we generally settle for an application form and an interview.

Why are we so resistant to testing and other more objective sources of data?

Perhaps it is because our expectations, preconceptions, and prior beliefs pretty much always influence our interpretation of new information. Experiments conducted over and over have shown that we see what we expect to see and conclude what we expect to conclude.

Tom Gilovich, a Cornell University psychologist, writes:

“Information that is consistent with our pre-existing beliefs is often accepted at face value, whereas evidence that contradicts them is critically scrutinized and discounted.”

I was recently helping a hiring manager evaluate his staff for promotion. We carefully determined success criteria and agreed that the candidates had to meet those criteria for consideration. In fact, we spent quite a bit of time validating the criteria and ensuring that others agreed with them. We then tested his entire staff using highly validated, widely accepted tests of ability and potential.

As you might expect, one of his favorite people did not do very well on any of the tests. He struggled with what to do and finally decided that his own judgment was more valuable than the tests and promoted her. Six months later she was not performing, was not happy, and he was now faced with the task of demoting her or letting her go.

This is just one example of some general truths. If we are presented with evidence from a test that a person we really like is not very good at whatever skill we are looking for, we say that he must have had a bad test day or we find some other excuse to downplay the tests results because we want to hire him.

On the other hand, if the person had not gone to the right schools and not worked at the right companies, or had displeased us in some way, we would be more likely to accept the test results as accurate.

There are countless examples of how we deceive ourselves in the process of interviewing and screening candidates. We ask leading questions to elicit the responses we want: “You have made presentations to senior management, haven’ t you?” We ask references the same kinds of leading questions.

It is not that we don’t examine information critically. In fact, experiments have shown that we look at all the evidence quite carefully, but we subtly massage it to make it support our preconceived idea or wish.

If evidence seems negative of our desire, we find excuses for why the information is bad or we lower it in our priorities for making a decision. We do just the opposite for favorable information.

We also will find data to validate our choices later on. If a person is successful, we will tend to attribute that to our superior interviewing skills, but if they fail we will find other reasons. Managers and recruiters are expert at the art of scapegoating their poor hiring decisions. What is most interesting, is how often someone removed from the process predicts the end result well before it happens because they see things more clearly and do not suffer the same preconceptions.

The bottom line is that assumptions, beliefs, and interviews are very poor tools for selecting people for specific jobs. It is almost impossible to apply objectivity to the interview process or to rid ourselves of deeply held (and mostly unconscious) beliefs.

Here are three things you can do to make you more effective as a selector:

1. Become aware. While we cannot prevent our preconceptions from clouding our judgment, we can apply corrective measures. We can develop criteria for jobs that are based on competencies, not on beliefs that cannot be validated or objectively supported. We can apply the scientific method to the recruiting process, just as we do to most other processes. I highly recommend How We Know What Isn’t So by Thomas Gilovich at Cornell, and Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart by Ian Ayres. They are eye-openers to how easily we are duped and misled by seemingly objective evidence and by our own human nature.
2. Learn that superficial, circumstantial evidence is probably misleading and often wrong. Every court of law has developed elaborate rules of evidence to ensure that they get as accurate and well-rounded view of a situation as possible. Even with all of those rules and procedures, many innocent people still get convicted. Finding objective criteria and evidence takes time and a willingness to seek it out.
3. Use objective tools such as validated tests and multi-rater feedback. By starting with one or two well-known tools, we can refine and hone them to our exact needs until they are excellent at predicting success. Proctor and Gamble has been doing this for more than two decades with remarkable success.

When it is essential that people learn rapidly and perform at superior levels, you need objective and repeatable ways to judge candidates. No scientist would rely on interviews, feelings, or opinions to judge a scientific experiment. Neither should we in judging a candidate.

Article and research by Charles Handler and Mark C. Healy

Over the past decade, there has been a significant increase in the number of assessment tools available for use in both hiring and developing employees at all levels of the organization.

By far the most common use of such tools has been for selecting hourly and technical workers. This makes sense, considering that the bulk of many organizations’ employees reside at this level and because success at many of these jobs usually boils down to a few key attributes such as:

• How fast can the person learn the job?
• Is the person going to steal, ditch work, etc?
• Does this person “get” customer service and will they help customers acquire what they need?
• Can this person communicate effectively?

As organizations gain more experience with measuring applicant qualifications, it has become common to find online assessment within the hiring process for hourly and skilled technical positions. Broader sales and marketing efforts by vendors and consultants have led to dramatically increased sales of these products. But what about managers and executives?

As we found in our latest survey of online prescreening and assessment trends, having candidates for supervisory roles (from 1st level all the way to the C-suite) complete questionnaires, tests, and simulations online is common, but there still seems to be a lack of assessment at these levels, especially when compared to the use of assessment for hourly jobs. Nonetheless, adoption of these tools has grown over the last 10 years and is likely to continue to do so over the next decade.

While the basic model for success with predictive assessments still holds, there are some key differences between successful assessment for managerial and executive-level positions:

• Managerial jobs require a slate of skills that are not often required for success at simpler jobs. These include things like leadership, business acumen, managing performance, strategic decision-making, conflict resolution, etc.
• A mistake at this level of hiring can be much more costly than for an hourly hire, mostly because managers are responsible for potentially hundreds of people and millions in both costs and revenue acquisition.
• Managerial hiring has a tighter integration with ongoing developmental strategy and activities as organizations look to maximize their investment in an individual by understanding and developing potential from within.
• There is a broad range of job complexity once you hit supervisory levels. The selection process for a front-line leader can differ vastly from that used for executive roles. Executives often complete day-long assessment centers while first-level supervisors are more likely to be presented with a simple in-basket or situational judgment exercise, which is scored automatically.
• Feedback becomes a more integral part of the assessment process. Pre-employment assessment for hourly jobs almost never provides candidates with any feedback at all.
• When making promotional or hiring decisions about managers, assessment requires a “whole is greater than the sum of its parts” approach. Trained assessors (often both external and internal members of the organization) make ratings based on a variety of data.

The above differences are clearly reflected in the past, present, and future of managerial-level hiring and development tools. Let’s start with the recent past.

The Past

Other than interviews, technical knowledge, and previous job experience, best-practice managerial assessment over the past 50 years has typically involved mimicry of the most important tasks faced by supervisors of both people and process.

Typical tools include:

• In-basket simulations. Also known as an “in-tray” outside of the U.S., these often-intense exercises imitate the administrative work of a manager in a fictional company. The traditional version of this tool presents a candidate with a stack of memos, a calendar, process descriptions, employee files, and other paperwork. The task is to respond to issues contained in the memos, with attention to the supplemental information about the company and its employees.
• Role-plays. Traditionally live and in-person, the candidate plays the role of a manager dealing with a poor-performing employee, dissatisfied customer, or other interpersonal situation. The interaction is scored (usually according to competency ratings) live or from a video or audio recording.
• Situational judgment. Usually in the form of multiple-choice questions, these assessments present a real-life work situation and ask the candidate to choose among alternative actions. Sometimes, multiple responses are correct, but each response is worth a different number of points.
• Business case/problem analysis. Some assessment systems require the potential manager to read over an operational or strategic problem and render an opinion or plan. Some include mathematical calculations and necessitate clear writing. Others essentially mimic a work-related intelligence test.
• Formal presentations. Particularly in the case of hiring at the director-level and above, and especially useful for selecting sales executives, the candidate is required to make a presentation about a product, strategy, or current issue. Aspects of the talk, including organization, clarity, and technical knowledge, are rated by assessors.
• Testing. There are several standard assessments that seem to be commonly used to provide insight into the personality and cognitive ability of managerial candidates. In many cases, the activities above get at the same basic constructs, but we have found that the higher the job, the more common it is to find testing as part of the assessment-center experience.

For higher-level jobs, the above ingredients are often combined into a one-half to three-day offsite often called an “assessment center.” Assessment centers are very expensive due to the need to have trained, qualified personnel to execute and score the exercises.

So, as the level of job gets lower, there is a decrease in the complexity of the assessment process.

The key factor uniting all of these diverse tools into an assessment center methodology is that they attempt to simulate managerial tasks and decisions in a real work environment.

Essentially a “test,” these highly regarded instruments ask a candidate to think and behave like a competent boss, instead of simply demonstrating a general trait (e.g., conscientiousness), skill (e.g., Intermediate Java), or ability (e.g., logical reasoning).

Moreover, these tools force the hand of more reliable, standardized data in a selection or promotional process, removing a worthwhile amount of subjectivity from key personnel decisions. There is even scientific evidence of decreased subjective bias on the part of hiring managers when compared with unstructured, ad-hoc interviews.

Another factor that underlies the use of managerial simulations is that they require some budget, effort, and dedication on the part of the HR and line organization, as does any worthwhile human capital initiative. Traditionally, these are expensive, somewhat cumbersome tools that are brought into a company by individuals who are just learning how to use them.

But simulations that realistically judge a candidate’s managerial prowess soon prove themselves to be invaluable to leaders making placement decisions.

Traditional managerial assessments have also demonstrated greater hiring accuracy. They’re more appealing to job candidates than personality inventories and job applications. They have a real relationship with future performance on the job when compared to other hiring tools. They are usually in the form of a statistical correlation with a measure of performance as a boss.

Decades of research in industrial/organizational psychology stands firmly behind these tools.

The Present

So what is the state of managerial hiring tools today? We contacted a wide range of assessment vendors to chat with them about the tools they offer for the selection of managerial to executive-level jobs.

Today’s market for these tools provides a continuum of options ranging from the old school to some thought-provoking new ideas. Here is a summary:

• Most assessment vendors and consultants understand that using the same content for lower-level jobs is not an acceptable strategy. Testing and measuring simple competencies does not account for the differences in what it takes to be a good entry-level worker and what it takes to be a leader/manager. We have seen an increasing number of vendors offering new products that leverage traditional approaches to making selection decisions for more complex jobs (i.e., assessment centers, in-baskets, etc.).
• Competency models, once thought of as a temporary fad, are now well-entrenched and help support linkages between employee selection and development. While competencies are all the rage for employee selection of all types, the link between selection and development tools for managerial-level and executive jobs is much stronger than it is for hourly jobs. Most managerial-level selection tools can also be used for development (with scores indicating strengths and areas to work on), and almost every provider offers a version tuned for both. This makes sense, as this kind of assessment is commonly used for identifying and grooming high-potential employees and for making internal promotional decisions.
• The in-basket is alive and well and living on the Internet. Most vendors offer an online in-basket that provides “assesses” with a realistic representation of a range of daily tasks required of someone at the appropriate job level. A simulation of an email in-box, coupled with organizational charts, employee histories, and process descriptions is most typical here, replacing the giant stack of memos with an MS Outlook-style in-box full of action items and hyperlinks supplemental information.
• Technology is being used mostly for delivery and administrative functions. There are many folks out there who are using the same basic content and format as has been used for decades. The biggest difference is that technology has been used to make the delivery of the content easier and to help make the tasks of scoring and report writing simpler as well.
• Online roleplay simulations are slowly starting to appear. We did see a decent number of products that provide the ability to use avatars and present assessees with more complex data and problems as part of the assessment experience. However, these are still well in the minority. Some of the technologies that will provide the true next generation of simulations do not yet exist. Forward progress in this area will be slow over the next few years.
• The more complex the job level, the higher the level of touch. Most of the standardized products we saw for the lower echelons of management use automatic scoring built into the product, whereas products for more complex and higher-level jobs use trained assessors to do scoring and feedback and act as role players.
• There is very little ability to customize content unless you have deep pockets. Most of the tools we saw did not easily support high levels of customization, though some firms sell versions of tools that differ by setting (e.g., healthcare, manufacturing, general office). However, there are different types of assessments for different types of jobs, and the tasks and competencies we saw are general for most leadership situations. As with assessment for individual contributors, you should analyze the job and carefully match assessment content to the actual work performed.
• There has been very little work done investigating the ROI of the online versions of traditional assessments. We still believe that asking someone to perform key aspects of a job is still the best way to evaluate how well they will do in their new role. Anecdotally, there are thousands who will testify to the impact of a good assessment program for selection and development of managers and executives.

The Future

So what does the future of managerial-level assessments hold? Here are a few ideas:

• More realism via simulations, particularly using avatars and online representation of complex, real-work environments.
• More artificial intelligence will be used in scoring, with technology that partially analyzes a written or spoken response by a candidate and, at the very least, speeds up the scoring and rating of completed materials.
• More data to support the value of managerial assessments.
• More diverse product offerings available as uptake increases in this hot area.
• Tighter ties with competency models will be created so that assessment and development will be linked more closely.

Conclusion

Today, best-practice managerial assessment involves a combination of the tried-and-true approaches of the past and the innovative media of the present.

For the organization that’s serious about leadership, managerial assessment tools (especially in-baskets and role plays) represent the key to truly separating those who aren’t qualified to influence one’s workforce from those who will aid revenue, retention, and overall performance.

We expect that as technology evolves, so will the automation of administrative- and scoring-related tasks and the level of realism associated with these assessments.

Article and research by Charles Handler and Mark C. Healy

For the last five years, Rocket-Hire has surveyed talent-management professionals who use Web-based screening and assessment products to enhance and quantify their hiring processes. Our goal for this research is to document trends in the usage of online screening and assessment tools in order to help provide a clear understanding of the popularity of these tools and their perceived positive and negative attributes.

Late last year, we again asked recruiters, HR staff, entrepreneurs, and hiring managers to tell us about their use and opinions of typical hiring instruments. This article highlights our major findings.

Today’s report is just a brief summary of the results of this study. For more data and an in-depth analysis of these trends, check out your May issue of the print publication, the Journal of Corporate Recruiting Leadership.

Overall, our findings from this year, and the trends that emerge over the five-year life span of this survey, continue to be positive. We had the largest number of respondents yet this year (141 recruitment and hiring professionals completed the survey, just a tad above last year’s 136).

The increase in use of screening and assessment tools and the positive benefits of these efforts continue to notate a strong trend toward their popularity and value.

On the negative side, results clearly indicate a continued lack of proper evaluation for these tools, which is even more discouraging when one considers that lack of understanding of the benefits of screening and assessment is still a primary reason for the failure to adopt them. Thus, the need to demonstrate the value of assessment by including evaluation as a core part of implementation is crystal clear and, in our opinion, remains the No. 1 challenge related to the use of assessment.

Now on to a more detailed review of our results.

Use of Applicant Tracking Systems

As with previous surveys, a majority of respondents employ an applicant tracking system. Specifically, 79% already have or will soon have one deployed, or are considering installation. As can be seen in the table below, no matter what the hiring demand, typical ERE readers are familiar with using an ATS.

Table 1. Use of Applicant Tracking Systems by Hiring Demand

Use of Prescreening Tools

About 57% use online prescreening to sort through their job applicants. We defined prescreening tools as:

Tools that gather information about, or ask candidates to respond to questions about their experience, skills, and qualifications in order to identify if they meet minimum job requirements. These tools are typically used early on in the staffing process.

The table below captures the usage rates of each of the most prevalent forms of screening.

Table 2. Usage rates of common prescreening tools

Common tools in use include the popular qualifications screen, technical skill certifications, and analyses of “fit” with company culture. Every year, it seems that more organizations are using prescreening as part of their online recruiting strategy. But the breadth of this practice varies, with some companies using these tools for all jobs while some use it for one or two alone.

Table 3. Extent of Prescreening Deployment

Use of Assessment Tools

In contrast to screening tools, online assessment tools were defined as:

Scientifically based screening tools that look more deeply into a candidate’s abilities, interests, and skills. These tools include personality measures, cognitive tests (i.e., verbal and quantitative skills), situational judgment tests, job simulations, etc. These tools are typically used for a more in-depth evaluation later on in the staffing process.

Underscoring the growing increase in the popularity of these tools, most common forms of online assessment showed an increase in rate of adoption over previous years, as comparisons with 2002 reveal.

Table 4. Usage Rates of Common Assessment Tools (2007 vs. 2002)

Specific skills and knowledge certifications and personality inventories are common assessment strategies once the pool of one’s qualified candidates has been reduced. Indicators of cultural “fit” and various cognitive ability measures are also widely used.

For another angle on the extent of assessment implementation, we asked respondents to indicate the level of jobs for which assessment is used for evaluating candidates.

Table 5. Job Level of Assessment Implementation by Job Level

2007 appears to be the first year that our data show a general pattern of use across all levels of an organization. Nonetheless, executive-level openings tend to entail more personal and direct methods of selection and placement, and were less likely to involve the use of any online tools.

Effectiveness of Screening and Assessment Tools

This year, 65% of prescreen users and 77% of assessment users felt their tools added value to their organization. In contrast, a full 21% of prescreen users and 10% of assessment users felt these tools did not add value. But this is difficult to judge, especially if one does not take the necessary steps to adequately answer the question. In this case, only 27% of users of prescreening tools and 36% of assessment users collected metrics to judge the quality of their interventions. Moreover, users of metrics tend to report success with prescreening and assessment.

Table 6. Perceived Effectiveness of Prescreening Tools

Table 7. Perceived Effectiveness of Assessment

Clearly, collecting metrics helps to answer questions about effectiveness and, interestingly, provide a positive response to these questions.

The Future of Online Screening and Assessment

Of those companies not currently using screening or assessment instruments, 42% feel they will implement one or more in the future. For them, Table 8 summarizes the sorts of tools under consideration.

Table 8. Screening and Assessment Tools under Consideration

As with the tools current users are seeking to use more of (or expanding to include in their hiring systems), qualifications screening, assessment of cultural fit, skill/knowledge assessments, and personality inventories dominate interest. Still, the mystery surrounding the value provided by these tools, as well as a prevalent intimidation factor, keeps companies from jumping in.

Obstacles to the Use of Online Screening

As usual, we asked users of online hiring technology (as well as non-users), what they perceived to be the biggest obstacle to the adoption of prescreening and assessment in their organization. Again, we received nearly identical responses to previous surveys.

Table 9. Single Biggest Perceived Obstacle to Deployment of Online Tools.

While 18% of users feel there are no obstacles at this time, many hirers feel that one or more roadblocks prevent prescreening and assessment from being integrated into the staffing process. These reasons mostly reflect a lack of knowledge of the benefits of more in-depth hiring techniques. Moreover, there appears to be a general lack of knowledge of safeguards and metrics designed to detect problems when candidate evaluation systems are in place, even among regular users of assessment.

But do these worries represent valid concerns, or is it simply fear of the unknown? Arguably, when assessment tools are properly matched to the demands of the open position, the only relevant concern is applicant identity (e.g., the stereotypical “smart older sister” taking a math test for an applicant) and test content security. But candidate enhancement of qualifications is common on resumes, in interviews, and for some personality traits/scales. Since many of the same concerns exist for paper-based processes, this should not limit any one organization from deploying Web-based hiring technology. Overall, most companies may legitimately cite few or no substantive reasons to limit their use of online recruitment and hiring techniques.

Summary

This year’s results let us know what’s really going on, and it confirmed what most of our previous surveys have found:

• Having an ATS installed is now nothing particularly innovative, especially if you work in a medium or large organization. For some, it’s hard to even imagine the old stacks of resumes and bulging file folders of applicant information.
• Prescreening and assessment are continuing their gradual penetration into the mainstream of recruitment and hiring.
• Qualifications screening, personality inventories, and skill and technical certifications continue to be the most popular online assessment tools. Assessments of cognitive abilities as well as fit with the company culture have expanded their footprint as well.
• Respondents report a lack of understanding, weak budgets, or a general lack of support for online tools as their primary obstacles to adoption or greater use of modern prescreening and assessment technology.
• Those organizations that formally evaluate their hiring practices tend to support the use of prescreening and assessment, but a large number of organizations are still failing to evaluate the effectiveness of their screening and assessment tools.

There clearly is a growing interest in scientifically derived hiring tools. This trend is encouraging but not particularly surprising; these numbers will continue to grow as more organizations understand the value in properly evaluating the impact of screening and assessment tools, seeing for themselves how quality hiring tools can improve a workforce.

As both authors have seen over a combined 25 years in screening and assessment consulting, skepticism about the value of these tools remains strong. We will continue to champion the use of evaluation to “close the loop” and reveal how more in-depth evaluation of candidates may positively impact the bottom line.

A word about our methodology: Respondents were evenly representative of recruiters, recruiting leaders, HR executives, business owners, and hiring managers, and represented a wide variety of organizations and hiring situations. Specifically, 59% were employed by organizations with 500 employees or more, with 34% working for a company with 5,000 or more employees. In contrast, about 20% were involved with recruiting for companies with 50 or fewer employees. In terms of employee acquisition activities, 34% hired more than 500 workers in a given year, but a majority of respondents (66%) made 500 or fewer hires per year.

The phone rings. Someone on the other end says he or she wants to build (or buy) a Web-enabled hiring test. Let’s say it will be for salespeople (generally the caller is a recruiter or HR manager, but sometimes he or she is a gopher).

After discussing the idea for a few minutes, I make a few suggestions. These always include following the ‘Guidelines’ to make sure the test is based on job requirements and business necessity and following the ‘Standards’ to make sure the test actually predicts job performance.

In almost every case, the caller is aghast at the work that needs to be done.

“All I want is a test!” they say.

“You want one that works?” I ask.

“Yes. But that’s hard!” they say.

“What’s your point?” I say.

On the other end of the phone, my keen bat senses hear muttering about me being a “jerk,” then dialing someone who will sell them the “mother of all tests”?one the vendor promises will work, regardless.

Why All the Fuss? A Test is Just a Test, Right?

The Guidelines and the Standards are not “nice to know” (i.e., limited to eggheads, legal eagles, and companies with U.S. operations). They describe how to define and evaluate job skills. That is, they first recommend test users define critical elements of the job based on job requirements and business necessity; then, they describe three ways to make sure test scores accurately predict performance (e.g., criterion, construct, and content validation).

Let’s reiterate. Step 1: Define job requirements and business necessity. Step 2: Make sure the test is predictive and stable.

Clear definition and evaluation is good for the hiring organization and good for the applicant. This principle works in all cultures and countries. So, if you plan to use a Web test, it’s a good idea to know the test actually separates qualified applicants from unqualified ones.

If you don’t get anything else, patch this into your screensaver: the only people who think it is too much work to follow best practices are people who don’t know how to do it.

But, no harm is done, right? Wrong. Highly effective hiring tests that claim they have no adverse impact or have been “validated” by the U.S. EEOC are as legitimate as the email announcing you won the lottery in Botswana.

Bad tests are really bad news for employer and applicant alike. A bad product backed by good-sounding marketing claims is still a bad product. And whether the user is in the U.S. or not, the test consumer, not the vendor, lives with the consequences of test use!

So, even if the vendor claimed his test was validated to grow hair on bald applicants, transform ugly employees into movie stars and cure morning breath, it would be your problem, not the vendor’s, to prove it.

Cause and Effect

There is a good reason why sailors advise passengers not to spit into the wind. The same is true for feces, fans, and bad tests. Eventually, even clueless test purchasers learn a weak test does not work as promised. You see a test that is not based on job requirements and business necessity, nor validated for the specific job, is designed to pass too many wrong applicants and fail too many right ones. It will show up on the job. That’s why the Guidelines and Standards are so valuable: they define exactly how to identify, qualify, and use a test that contains the least amount of error.

The bottom line is no matter how many years a person has been a recruiter; no matter how smooth his or her marketing campaign; no matter how certain he or she is about being a recruiting expert; and no matter how famous their organization, the ‘Guidelines’ and ‘Standards’ set the bar for measuring job skills.

Let’s examine how the ‘Guidelines’ and ‘Standards’ work for a sales position.

Sales Hiring 101

First, any method of separating qualified from unqualified applicants is a test. And “assessment” is just another word for “test.” We assess resumes, application forms, and applicant skills. The vast majority of organizations, unfortunately, use a two-step assessment process. Step one: use an interview to screen out most of the riff-raff. Step two: let the job screen out the rest. The two-step process explains in large part why 20% of salespeople generally produce 80% of the sales. Only riff-raff were screened out pre-hire.

Screening out riff-raff is easy. All you have to do is get to know the applicant, examine earnings statements, and dislike his or her personality. Normally, organizations screen-out 3.5 applicants to get one promising employee. On-the-job performance screens another one of two. Over time, this makes the final hiring ratio about 7 to 1. Riff-raffing is the norm and riff-raffing is expensive.

Let’s look at the cost of using the job as an assessment in terms of training, travel expenses, management coaching, and salary for six months. We’ll be conservative. One week training = $2,500; sales travel expenses = $100/day for six months or $12,000; coaching time = 15% of manager’s time or about $6,000; and, six months’ salary and benefits = about $36,000. This totals about $56,500 per salesperson (ignoring recruiting fees, lost customers, empty territories, and so forth). Bottom line? In round numbers, riff-raff assessment costs upwards of $50,000 for each lost salesperson.

Error-Free Hiring?

Mistake-free hiring is pure fiction, but doing a better job screening is not. First, you have to fully understand your specific sales job and the critical skills that separate the successful from the unsuccessful salesperson. This kind of information is seldom obvious. It does not come from generic tests, averaged scores, and calculating group norms. Generic norming is bad science. It serves as an example of wrong-headed test practices.

A trustworthy and reliable test involves in-depth understanding of critical job functions, measuring every critical skill area at least twice, doing a formal study to confirm scores predict job performance, and monitoring adverse impact. In professional terms, this is called job analysis, validation, multi-trait-multi-method assessment, adverse impact monitoring, and continuous improvement. If it sounds like a good way to do business, it is. If it also sounds like hard work, it is.

In the next few paragraphs I’ll briefly describe what to look for in a sales selection system.

Professional Job Analysis

As mentioned above, a professional job analysis does not consist of giving everyone a questionnaire and comparing top-performer scores to bottom performers. This is the first sign of buyer-beware because it makes some huge and often wrong-headed assumptions.

It assumes an equal playing field. That is, all productivity results are equivalent. New accounts, customer service, market conditions, and expanded accounts are all rolled-up into the same category: productivity. In some cases, overall performance might even be complicated by (gasp!) skillful manipulation of numbers. Separating salespeople into top and bottom producers based on sales dollars is a sure clue the analyst does not understand sales.

Suppose you are like most folks in the hiring business and you expect your test to accurately predict job performance before you commit big bucks to salary. By definition, your test should measure something that causes performance. If you give one big test to everyone without knowing explicitly what you want to evaluate, you fall into the “correlation or causation” trap. As an example, ice cream sales and shark attacks have a strong positive correlation. Does that mean shark sightings cause people to eat more gelato? That Ben and Jerry’s Chunky Monkey is a poor shark repellent? Or perhaps sharks have a seasonal business they don’t want people to know about? Homegrown questionnaires often confuse correlation with causation. Just remember: Unless water-born ice-cream is proven to attract sharks, one does not cause the other.

A good job analyst knows how to identify key skills that make the difference between successful and unsuccessful cold calling, repeat sales, strategic selling plans, customer service, and so forth. In many cases, they may involve totally opposite skills. Treating sales production as a discreet measurement point is like putting fruit salad in a blender, pressing the annihilate button, and testing the puree for peaches. A professional job analyst knows key information can only come from people doing the job, not from supervisors or aggregated production data.

Let’s say the analyst has done his or her homework. Now what? The hiring manager does not have weeks or months to evaluate applicant skills. Unless the hiring manager uses the hire-and-hope strategy, sales skills have to be evaluated in minutes or hours. If we have done our job right, we will know the mini steps that lead to maxi results.

Bottom line? If the analyst asks you to lump producers into groups and gives them all the same test, you are about to see your money pour out the door.

Does the Test, Test?

The only test that is worth anything is one that works for your job in your company, not one that worked for the company across the street, or a job with the same title, or matches a nationwide norm, or even a company in the same industry. It has to work for you.

Sometimes a validity study can be transported from one job to another, but that is only if you know for certain the two jobs are essentially the same. But if the market is different, the company environment is different, products and services are different, customers are different, or sales cycles are different, then how can any reasonable person claim XYZ scores predict cold calling, customer service, or sales expansion for your position based on one that is entirely unknown? Doesn’t that seem a little far-fetched to you?

The only time you can trust that another test will work best for your organization is to compare the job analysis from the other test to the job analysis for your job. If the two jobs are essentially the same, then use it; if not, you “pays your money and takes your chances.”

Give a generic personality test to salespeople and see what shakes out? Get ready to see a great big pile of belly-button lint.

Our Test Does Not Discriminate

In the U.S., at least, large organizations and Federal contractors are not supposed to reject qualified applicants based on age, gender, race, and so forth. This is called discrimination; but there is something else called adverse impact. What does adverse impact have to do with discrimination?

The legal definitions have subtle overlap, but for the purposes of this article, let’s assume discrimination generally means that an organization intentionally discriminates against certain kinds of job-qualified people?in hiring, promoting, training, and so forth. While adverse impact generally means the hiring system, even though it is job-related and professionally validated, unintentionally discriminates. In lay terms, think of discrimination as intentional and adverse impact as unintentional. For any better definition, see your local labor-law attorney to explain the details.

I consider discrimination unethical. Everyone deserves a chance to work in a job for which he or she is qualified. But here is where things get complicated. Government agencies examine discrimination at the group-level. Hiring managers don’t care much about group performance. They care about individual performance.

This raises a problem that all hiring professionals need to consider. By way of example, suppose 200 people apply for a job. One hundred are Lilliputians and 100 are Yahoos. At the group level, 70% of the Lilliputians are hired, while only 40% of the Yahoos make the grade. At the individual level, there are quite a few Lilliputians who are miserable workers, just as there are quite a few Yahoos who are top performers.

From the organization’s viewpoint, they only hired job-qualified people. From the government’s viewpoint the company discriminated against the Yahoos.

Who’s right? It’s hard to tell, so the government examines the organization’s:

• Professionally developed job analysis (to show hiring tools are based on job requirements and business necessity)
• Professionally conducted validation study (to show hiring tests and interviews accurately and consistently predict performance)
• Pass and fail results for Yahoos and Lilliputians at each step of the hiring process
• Proactive efforts to develop tests with less adverse impact on Yahoos

As long as the company has done its homework and followed generally accepted hiring practices as outlined in the “Uniform Guidelines” and “Standards,” it is not in trouble and will have hired all the best and most diverse applicants.

So what’s the problem? Some vendors claim their tests have no adverse impact. But research consistently shows hiring tests for jobs requiring problem-solving ability almost always does have an adverse impact when examined on a group level. Competent test vendors know this. Incompetent ones don’t.

Automated Resume Screens

What recruiter or hiring manager has not seen a brilliant resume developed by a blatantly unqualified candidate? And what about the marginal resume presented by a remarkable applicant? At best, a resume includes Kodak-moments recalled by the resume writer. At worst, a resume is an exercise in creative fiction.

Think about it. Every applicant is motivated to write just enough words to garner an interview. While every hiring manager wants to find someone who was an exceptional performer in the exact same job at another company doing the exact same work. Generality goals meet specificity objectives.

Sophisticated applicants know how to pepper the resume with keywords and qualifications that may be fact or fiction; different hiring manager’s screen resumes using totally different criteria for the same job; and, everyone makes massive inferences based on snippets of data. So. Tell me again. Other than keeping a few programmers in work, what is the benefit of automating resume searches?

Back to the Beginning

So here we are, back at the beginning. Tests are abundant. And if all you want to know is a score, anyone test will do. Good tests, however, ones that accurately predict job performance, are rare. You can trust a good test to produce good employees. You can tell the difference by following a few guidelines.

Avoid vendors that emphasize their non-discrimination aspects, “legality” or industry-wide applications. Assuming their claim is accurate (and I have yet to see one what was) users are responsible for their own test use. Vendors are off the hook.

Avoid vendors that want to give their test to two groups of producers and use the results to predict job performance. These represent bad science. Scientifically, this kind of study can only show whether the two groups are different, but it does not tell you why. And it does not tell you about individuals within the groups.

Avoid tests that are based on self-reports. Self-reported answers can be faked. They cannot be validated by outside sources. Self-reported tests are similar to resumes. They represent things the test-taker wants you to know about him or her. Making decisions about hard skills based on self-reported data requires a huge leap of faith that is generally wrong half the time.

Ask the vendor for a report showing he followed the ‘Guidelines’ and ‘Standards.’ This is your only assurance the test will be job related, based on business necessity and accurately predict job performance.

Web-based testing is in the same category as medicine was 100 years ago when heroin was good for you; there was no such thing as anesthesia; injections were unavailable; radioactive water cleared the mind; opium was a relaxation agent; blood-letting was commonplace; linseed, mustard, and soap were used as cure for infection; and sugar of lead was a common treatment for diabetes.

Let’s all work hard to move hiring into the 21st Century.

Every year, I take a few minutes to reflect on the things I have noticed while working with both producers and consumers of screening and assessment tools. Overall, I am very encouraged by what I have been seeing. The market for screening and assessment tools continues to grow. This makes me extremely happy because we I/O psychologists know the value that is to be had via the use of quality assessment tools.

The science geek in me is also very happy to see strong investment in innovation. I am really pleased to see the ways in which quality content is being combined with technology to collect the mountains of data that are required to uncover underlying truths about the relationship between human traits and job performance.

One of the most valuable means of insight into assessment trends over the past five years has been the annual ERE/Rocket-Hire Screening and Assessment Usage Survey. If you have not taken the time to complete this survey yet, please consider taking five minutes to help us out. Your input really does make a difference, and we greatly appreciate your time. A link to the survey is provided at the end of this article.

The major theme for 2008 will be increased integration of assessment into carefully designed products. Much of the rationale behind these products is the reduction of the work required to implement assessment via a combination of data and technology that results in a turnkey product that is both relevant and easy to use. This trend is going to continue to increase the accessibility of assessment and lower barriers to entry that have long been limiting the use of assessments. A quick look at the trends outlined below clearly demonstrates support for this point. In 2008, I believe we will see the following trends continue to unfold:

1. Continued Interest and Use in Assessment. The market for assessment tools will continue to increase as it has every year for the past decade. This will be the result of a number of factors, many of which are discussed in more detail below. In my mind, however, the number one reason for increased adoption is still the fact that properly used screening and assessment is one of the most important ways to increase the quality of hiring decisions.
2. More Vendors to Choose From. As interest increases, so shall the number of options available to consumers. For the most part, this is a good thing, as the bulk of new companies and offerings are being created based on solid best practices. As always, there will continue to be less sophisticated or technically-sound options available. With so many options to choose from, extracting the most value out of assessment tools will continue to require a solid game plan and due diligence as the foundation of the vendor selection process.
3. Increase in Acquisitions for Content. As companies continue to work toward developing products and increasing use of assessment tools, many are discovering that it is easier to purchase content that has a strong history behind it than it is to create their own. This has also been playing out as existing assessment companies continue to purchase smaller companies in order to increase their product range. Recruitment Process Outsourcing (RPO) is also a huge driver of this trend. RPO will continue to be a major force in the future of hiring, and assessment should end up as a key part of the better RPO offerings.
4. Increased Integration and Packaging of Assessment into Off-the-Shelf Products. Directly related to the trends mentioned above, there has been a continued push towards the development of product suites that provide a hiring process into which assessment is embedded. The goal is to create a more turnkey product that can be lightly customized and quickly implemented. The more quality assessment that is baked into a well thought-out process that can be implemented relatively easily, the more quickly assessment can begin to return value. This equation also lessens the need for professional services and makes the product a bit more “foolproof.” The more turnkey the product, the less assurance one has that it is “dialed in” to a particular position or job. The decision to use a more turnkey product as opposed to a more customized approach should be treated as a business decision driven by the contingencies of one’s particular situation.
5. Increased Attention to Products/Sales into Vertical Markets. Again, as the trends noted above unfold, we are seeing an effort to package assessment products that have been developed specifically for certain vertical markets. This is driven partly by the fact that vendors are finding that it makes sense to promote and sell products in a more targeted manner. Health care is one of the hottest examples of this trend, but I have also noticed increases in product creation and promotion in manufacturing and financial services as well.
6. More Products Designed for the Middle Market. The labor-intensive nature of assessment tools has not traditionally lent itself to use by smaller or mid-sized companies. An increasing number of vendors are creating turnkey products designed to overcome the need for professional services and large-scale data collection exercises. As interest in assessment continues to grow, these products will be well positioned to provide value for a wider range of companies.
7. Movement into Job-Search Process. An increasing number of job boards/career portals are discovering that adding assessment can provide much better search/match functions than do traditional methods, such as keyword search. I have been pushing this idea for the past five years, but this year will provide a noticeable increase in the use of assessment tools as part of the DNA of many job boards.

While the above represent what I feel to be very positive trends, I also see the perpetuation of a few trends that I am not so wild about. These things include:

8. Continued Fragmentation. Most users of assessment seem to still take a test-focused viewpoint in which the emphasis is on the use of one specific test rather than on the result of a coherent process. While some situations warrant this type of approach, it is best to build a process in which assessment data is tightly integrated into other decision-making data. The good news is that all of the trends outlined above represent movement in the direction of less fragmentation.
9. Tactical Focus. Assessment is still being used in a highly tactical manner. It is most often considered as a way to fight fires, as opposed to a preventative measure. As consumers experience more success with the use of assessment tools, a more strategic focus will result. It will be some time before we see this type of usage occur, but we are headed in the right direction.
10. Continued Lack of Proper Evaluation. My head hurts from repeatedly banging it against the wall over this issue. Yes, it is hard to create the buy-in needed to properly evaluate the impact of assessment tools. However, without proper evaluation, it is very difficult to clearly demonstrate ROI and build a solid business case for the use of assessment tools. While vendors are trying to help design products that can assist in this area, proper evaluation requires commitment from the consumer. The consumer is the one who needs to collect the data relating to the business impact of assessment. While this is not an easy task, if we can put a man on the moon, we should be able to collect relevant performance data.

This promises to be a great year for screening and assessment. Continued interest by consumers has created a buzz amongst vendors who are, in turn, creating products designed to eliminate the headaches often associated with using assessment. Of course, there are always tradeoffs. The more turnkey a product is, the less it will be optimized for a particular local situation. The degree of customization required is a business decision that should be made based on a thorough needs and cost/benefit analysis. Some things never change!

Want to help us learn more about what is going on with the use of screening and assessment? Please take five minutes to complete our survey. We will be sharing the results with the ERE community this spring.

Get used to it: unless your organization hires everyone who applies, you are testing. Some people (even attorneys who should know better) vigorously deny that their organizations test applicants (pssst?interviews are tests!).

Whether an organization uses verbal questions or written questions, they both have the same objective: to separate qualified applicants from unqualified ones before spending big bucks on salary, benefits, and potential lawsuits. Tests are tests.

Now?let’s discover whether your test is working for you.

A Good Test

Separating a good test starts with reliability. Suppose an applicant takes a test on Monday and on his way out, you deliver a carefully aimed blow to the head sufficient to cause short-term memory loss (but not permanent damage).

After he gets out of the hospital, you invite the applicant back to take the same test a second time (with the promise of safe passage). Will he score roughly the same? That is, can you trust the scores to remain consistent from one time to the next?

This is called “test-retest reliability.”

Reliability means you can trust a test to deliver similar scores regardless of when it was taken. Otherwise, you would never know whether it was accurate.

Interviews, for example, are notoriously unreliable. Interviewers tend to like or dislike applic