High-Level Observations

• The rise of talent management. This seems to be the age of  “talent management” when it comes to the use of technology in HR. I saw a ton of companies offering “talent management systems.” These platforms use technology to cover a broad footprint of key HR areas/functions such as branding, recruiting, onboarding, learning, development, and communication. Talent management products are starting to provide HR practitioners with a technology based backbone that will allow them to integrate major HR functions. The integration of more functions into one platform is a trend that can have significant value given the traditional walls that tend to exist between the major areas of HR in larger organizations. There seems to be variation in the functions offered by the various talent management platforms as well as some gray area around what defines a talent management product. These gray areas are nothing new, and are indicative of the nature of an industry-wide trend that is causing vendors to jump on the bandwagon. While my overall thoughts about talent management products are definitely positive, one wonders how many vendors have just dubbed themselves as “talent management” providers to be trendy, and have not significantly changed their products.

• ATS Not Hip Anymore. Is it just me, or does there seem to be a significant decrease in companies selling products referred to by the term “ATS”? I could not help but think that much of the momentum in the use of hiring and technology seems to be in the idea of the broader, strategic, idea of talent management and less in the more tactical area of applicant tracking. The concept behind applicant tracking and its related functionalities are still of great importance; it just seems that the term itself is losing favor as traditional ATS functionalities are being baked into other products such as advanced screening/assessment management platforms and talent management systems.

• Goodbye paper resume. It seems that there is continued movement toward removing the resume from the hiring process. This is definitely true of the paper resume, but there is a broader movement afoot to deconstruct the resume and extract the types of data it usually provides (i.e., skills and experience). Key information about candidates is being collected via other opportunities in the search and application process such as the creation of detailed profiles that have fields that guide the entry of important candidate information in a standardized format. Parsing, another method of deconstructing resumes, seems to be continuing to evolve as a means of extracting important data from resumes. Resume deconstruction methods allow for much more efficient searching, and matching between candidate data and important job requirements. These methods are part of an overall trend that will see the integration of a variety of candidate information into a digital platform that will allow it to be standardized, categorized, and compared to key requirements for a job or career. While we have a long way to go toward the complete death of the resume, technology is helping us to continue the slow march towards the inevitable.

• Web 2.0 is hot. It almost goes without saying that the latest in HR technology is leveraging the benefits of easy access to information, communities, and data that are at the core of Web 2.0. Web 2.0 was everywhere at this show. It seemed to me that the applications of web 2.0 seemed to be concentrated more in the space of performance management products, but there is no doubt that it is having a large impact on the thinking of those creating recruitment-oriented products as well. I saw a heavy focus on the use of social networking tools such as Twitter and Facebook and on products that provide companies with platforms on which to build communities. The data-driven nature of web 2.0 provides a good deal of promise for the area of recruiting and hiring.

• We’ve come a long way. The products I saw at this show provide a sign of continued progress and evolution that seems significantly more advanced than the products available five years ago in many ways. While the core function of many products may remain the same, we are continuing to develop new ways to connect people and information. The products I looked at seem slicker, cleaner, and more usable than ever before. While it is often hard to cut through the smokescreens when one is looking at products on the tradeshow floor, the overall level of tech-savvy seems higher than ever. I was able to see firsthand the positive impact of technology on a variety of recruiting- and hiring-related products. Of course keeping up and separating faddish technologies from those that truly are game-changers will continue to be a challenge in years to come.

Assessment-related Trends

• Assessment is still an outsider. While there were a decent number of assessment vendors at the show, as an overall area of HR technology, assessment is still a very small piece real estate. This makes sense to me given the problems that organizations seem to have in understanding the value assessment can provide.

• Two directions of evolution for assessment. Technology is helping assessment to slowly evolve in two directions. The first of these involves assessment being integrated into the functions provided by bigger, broader systems and products such as job boards, ATS, and talent management. Embedding assessment into other products makes sense when one understands that the core value of assessment is to provide information that can be used in conjunction with other information to support informed decision-making. Pre-employment assessment actually may have the most value when it is an integrated part of a process-based approach into which its results are integrated. This trend has been very slow to develop, but we will continue to see a trend toward embedded assessments. The second direction in the evolution of assessment is toward the productization of assessments to be sold transactionally. While assessment has been sold transactionally for decades, the present state of evolution leverages millions of data points to help provide a new level of clarity about what content predicts certain traits, behaviors, and outcomes. This evolution allows for off-the-shelf products that are more accurate than ever before, while providing documentation to support the relevance of these products for specific jobs.

• Hello, talent management providers? Assessment seemed to be conspicuously absent from the slate of services provided by most talent management systems. While assessment is part of the deal for some of the big players in talent management, it does not seem to be a core part of the concept at this point. This is upsetting given the value assessment can have, not only during the hiring process, but throughout the entire employee life cycle. Anyone who considers themselves to be a player in the realm of talent management should strongly consider embedding quality assessment tools.

• Platforms evolve nicely. The software platforms that accompany pre-employment assessment products are evolving nicely. I refer to these as “candidate management systems.” Today’s candidate management systems offer many of the functionalities once seen only in ATS products. This evolution should help support the value proposition of assessment as products become easier to use and provide additional decision-making support.

• Simulations still lacking. While there has been some definite positive movement toward the creation of more advanced simulations, we still need to be pushing ahead in this area. While there are some nice simulation products currently available, the future of hiring will continue to move toward the increased use of simulations. Not enough steps are being taken toward the development of truly innovative and unique simulations. This is an artifact of a lack of buyer interest and the absence of “killer apps” that are needed to allow simulations technology to advance.

When it comes to HR technologies that support the recruiting and hiring process, we are moving in the right direction and have come a long way over the past five years. The technology level available for assessment both as a core process and as a supporting cast member is more advanced than ever. However, there is a lot more that we can be doing to integrate assessment into the products that seem to be marking the trends in HR and technology. Assessment still seems to remain the misunderstood stepchild of HR, which is a shame since there is so much evidence for the value it can have. I remain optimistic that assessment will continue to be brought more tightly into the fold in years ahead.

But many folks predict that it’ll be increasingly harder to find qualified applicants to fill job openings, especially when it comes to white collar, managerial, and professional-level jobs. We have all dealt with numerous situations where the age-old “mirror test” (hiring anyone who is breathing and thus has the ability to fog a mirror) is the only thing required to fill a position. In such cases, it’s harder to make an argument for using something that will actually serve to further reduce the number of applicants to be considered.

Assessment can have value even when one has few applicants to chose from or even when there is a shortage of qualified applicants. Below are five good reasons why it still makes sense to use assessment, no matter what the labor market looks like.

Reason 1: Assessment can be part of a good sourcing and branding strategy

Assessment has begun to occupy new places within the hiring process. Over the past five years we have seen it begin to be folded into the job searching and matching process. Assessment is a great way to help match job seekers to openings that are a good fit for their background, skills, and values. Even in a tight labor market, job seekers can still benefit from some direction to help them make important decisions about what jobs and organizations are right for them.

Compelling employment branding is the first part of the equation here. Much employment branding seems to be a regurgitation of the same old song and dance. What company doesn’t value diversity? What company doesn’t care about the environment? Assessment can help us to get past the fluffy, generic employment branding stuff and get right to the heart of the matter: matching people with the right employment situation.

Recently, assessment has become a key ingredient in “values matching,” a branding and sourcing strategy that can really make a difference. Values matching will continue to be a hot area because job applicants wants to ensure they “fit” with the culture and value of the organization. Assessment is an excellent way to provide a index of “fit” that can be used to help applicants to understand how well what they may fit with an organization, or even with a specific role, job, or workgroup within that organization.

Organizations that fill their hiring funnel with applicants who possess congruent values and have some degree of fit will find it easier to be sure they hire persons who have a good chance of sticking around longer and being more productive. Values matching can be used as part of an initial sourcing strategy by including it as a key element in the creation of job profiles used to match applicants with openings or as part of the search process on corporate web sites. Even if there is only one applicant for a position, if that applicant is a good fit, hiring them represents a good decision for all parties involved.

Reason 2: Assessment can still provide insight needed to support good decision-making

Suppose there are only two applicants for one open position. Whomever is making the hire still must make a decision between the two applicants. The decision-maker can still benefit from having some data to better understand each of the applicants from which they have to choose. Most employment decisions are made using resumes and unstructured interviews. These are the two most common tools available to hiring personnel. While both of these tools do have some value, they function best when used in conjunction with information that can help those doing the hiring to read between the lines. Assessment is a perfect tool to help those making hiring decisions to better understand each candidate. Informed decision-makers are more likely to make accurate decisions than those who must make decisions in the absence of good data.

The results of even a basic assessment can provide decision-makers with additional data points to help them make a more accurate decision. The number of applicants one has to choose from does not alter the idea that informed decision-making is the way to go. So, even in a tight labor market where one is lucky to have even a few applicants, it’s still valuable to provide decision-makers with tools to help them better interpret subjective information such as resumes and unstructured interviews.

Reason 3: Assessment can help you hire for potential

One common strategy in tight labor markets is to shift focus from hiring for a specific set of skills, abilities, knowledges, etc. needed to do a specific job toward hiring for potential. In such cases a few key abilities or characteristics are identified and applicants are evaluated based on their ability to bring this valued “raw material” to the table.

This change in focus allows the organization to hire those who may not have had the exact experience needed to do the job for which they are applying, but who have what it takes to learn the job. Such a strategy requires a strong focus on training and development. Still, such a program allows organizations to take a broader focus when looking to fill open positions. College recruitment and hiring programs are an excellent example of this. Hiring individuals straight out of college allows companies to fish in a bigger pond. Assessment is an excellent way to assess potential. Basic cognitive ability and problem-solving assessments provide an easy way to identify individuals who have a basic set of tools that will allow them to be an asset to the organization. Setting up such a program can often be less complicated than developing an assessment program that is tied to a specific job. There are tons of good quality, off-the-shelf assessments that have been created with the specific goal of measuring general constructs. These can be plugged right in, with less up front work than may be required to create a job specific test battery.

The idea of hire-for-potential, train-for-success is an attractive proposition in a tight labor market.

Reason 4: Assessment can support onboarding and development

Assessment is also a valuable tool because it can provide a good initial picture of an applicant’s developmental needs.

Most assessment providers have the ability to create an initial development report based on the results of an applicant’s pre-employment assessment. Even if there are few applicants for a specific job, using an assessment as part of the hiring process helps the new hire hit the ground running. Assessment data can help with onboarding by allowing the new hire’s manager to have a good idea of developmental needs on day one. This information can be used to create an initial development plan and to provide a baseline for future performance management and development activities. Such a strategy can also have a positive impact on things such as commitment, satisfaction, and turnover. Research has demonstrated that good experiences during the onboarding period can have a positive impact on each of these important outcomes.

Reason 5: Assessment can help the organization to better understand itself

You can’t expect strong results from the use of assessment without a clear understanding of what needs to be assessed, and why. Usually, this involves spending some time to take a look at both personal and organizational factors that must be addressed using assessment. More sophisticated implementations involve the ongoing use and development of an organizational competency model. Such a model allows the organization to define the key elements required for success across all jobs as well as for each specific job.

While this is not always easy, taking the time to break jobs down into the components required for success provides the organization with important insights about what is required for success. This information is valuable for hiring, but it also has value for other important processes such as development, training, and succession planning. Going through the effort required to anchor a good assessment program has value because it helps the organization learn about itself. This learning extends beyond the ability to identify key determinants of success. Using assessment properly requires a good bit of learning and experience. Why shy away from assessment during a time that can provide valuable hands-on experience? By choosing to use assessment in good times and bad, companies can gain experience that will benefit them in the long run.

No matter how many applicants there are for a given position, hiring the right one should be the result of an informed decision making process. The information used to support this process should be directly related to the various things required for success at the job and the organization. This is a universal truth. Those organizations that begin to cultivate this mindset will realize a significant long-term advantage that extends past the value of good hiring, into the realm of developing, managing, and retaining talent.

In this article, I praised the progress being made in understanding the value of quality assessment products in the modern hiring process.

I strongly believe that the words I wrote in 2006 are as relevant today as they ever were; in fact, they are more relevant now than ever. The days of the big job boards and their keyword matching strategies are numbered.

A lot has happened in the world of online recruitment in the last decade. The focus of most of the evolution has been on candidate quality and on increasing accuracy. This is in opposition to the focus a decade ago, which seemed to be on quickly delivering a large volume of candidates. What we have learned since then is that without a way to quickly evaluate candidate quality, we end up saddled with information overload. So, the objective for recruiting products is shifting to one that is focused on the alignment of personal characteristics with those that are required for performance on the job. This is a good thing. This movement has definitely been aided by the choice of several leading career portals to embed candidate quality into their bag of tricks.

A slice of the content of the original article from 2006 appears below. It has been updated to include a few examples of a few career portals that are now using assessment to help them better serve their clients (both job seekers and employers) by providing them with a way to quickly focus on candidates who have what it takes. Before we get started, I want to offer a few disclaimers here.

1. I am not saying that using assessment as part of the job search and match process provides all the data needed to make a good hiring decision. Rather, I am saying that it provides an excellent way to address the idea of “garbage in, garbage out.” By creating an applicant pool of individuals who are at least qualified and motivated, we are taking a very important first step towards ensuring systematic quality in the hiring process. I think of it as pre-screening that comes before pre-screening because it provides an applicant pool that will give one much better odds of making a successful hire.
2. In this article I am going against my policy of not ever naming the names of any vendors who provide the type of products discussed. Please note that none of the vendors discussed here have been included due to anything other than my personal opinion that they are good examples of the trend I am documenting. I was not compensated by them, nor does my inclusion of them in this article mean they are the perfect solution for your problems.
3. Although I am an expert in this space, I do not know everything, and there is definitely a chance that I failed to mention a company that provides the exact type of service I have described. I encourage someone from such companies (or a fan of one of these companies) to contact me to let me know what you are doing.

Below are some relevant highlights from my original article.

Folks are finally starting to get the idea that assessments are but one component of a broader process designed to help hiring professionals make systematic predictions that result in good hiring decisions. There’s much to be gained from this process-oriented approach, so it has been great to see assessments being used to help at various phases of the hiring process.

Despite the continued integration of assessment, there’s one area in which assessment has been underused. This area is the use of assessment tools during the job-searching process. Specifically, the integration of assessment tools into the searching/matching component of career portals.

Traditional methods available to users of career portals for locating jobs are extremely crude, consisting mostly of keyword searches or matching based on simplistic profile elements.

The fact that this basic process has endured as the standard for almost a decade now clearly reflects the quantity-over-quality focus that has reflected the marketing-centric attitude of many job boards since day one.

Blocking Out the Noise

This crude matching process results in excess noise. For applicants, noise means they’re presented with, and encouraged to apply for, a ton of jobs for which they’re not qualified. For hiring professionals, noise means no mechanism to assist them in making an initial high-level determination of applicant quality.

Noise makes it difficult to make good hiring decisions because it greatly increases the chance to make systematic errors.

The good news is that I am starting to see a shift in focus that will go a long way toward a reduction in noise and an increase in the ability to differentiate candidates based on various quality factors.

This shift involves the use of assessment tools as an integral part of the services provided by career portals. The use of assessment to help fill the hiring funnel with applicants who have the attributes required for success has numerous advantages, all of which are explained by simple probability theory.

Think about it: the more qualified the individuals in your applicant pool, the better chance you have of hiring someone who has what it takes. Over the past few years, I have learned of an increasing number of companies who are seeking to change things by integrating assessment into the matching process.

The basic aspects of this include the following:

• Seekers create profiles. This part of the matching process works as part of a registration process, before a job seeker has even expressed interest in a specific position. Along with other information collected during this process, the job seeker creates a profile based on a short assessment of basic qualities such as work attitudes, personality measures, etc.
• Corporate users define high-level requirements. This part of the matching process requires hiring personnel to create a template of the basic things it takes to do the job well. This often includes standard things such as skills and experience but also includes qualities often measured by assessments (personality factors, dimensions of fit, work values, etc).
• Noise is squelched. When a job seeker searches for a job using the profile he/she has created, results include jobs for which they are suited based on the compatibility between their profiles and those of available jobs.

There are many twists on this formula but the basic idea is the same: deliver an applicant pool that has less noise and makes the recruiter’s job easier. This is the start to ensuring quality hires because it’s sure hard to hire superstars if they aren’t even in your applicant pool to begin with.

Admittedly, my description of this process is overly simplistic. In job matching, just as with making hiring decisions, assessment data should serve as one of many data points that are used to identify the best candidate for the job. Truly effective matching systems will use parameters that include other key determinants of success such as experience, knowledge, and skills.

Weighing the Pros and Cons

Overall, the use of assessment in the matching process offers the following five advantages:

• Reduces noise. Allows filtering based on actual job-related parameters.
• Helps determine corporate fit. This process is a good way to find and hire applicants who fit a company culture.
• Increases quality. Allows insight into applicant’s ability to deliver what it takes for success.
• Educates job seekers. Allows applicants some insight about themselves and how suited they are for a particular job or career.
• Results in branding. It can provide a good applicant experience, leading to support for employment brands.

Of course, there are potential drawbacks to the use of assessment as part of the matching process. These include the following five issues:

• Learning curve. Users must learn to understand how to define jobs in terms of relative importance of the parameters used for matching.
• Accuracy. It is critical to match the templates against which applicants are matched to accurate reflections of the job.
• Uncertainty. Predicting performance is a difficult task, to say the least. Assessment-assisted matching is not a panacea and does not eliminate possibility that mistakes will happen.
• Quality control. To be effective, assessments must be created using the proper methodology; substandard assessment products will produce substandard results.
• Legality. While the use of assessments in matching is not illegal, it does require awareness of some potential legal ramifications (see original article for full coverage of legal issues related to the type of products discussed in this article).

So who is actually using some sort of assessment as part of a career portal or job board? I believe the companies below are all doing great things to move us forward toward an era of embedded assessments that can help reduce the noise associated with finding and applying for jobs online.

Climber: Climber.com uses some really cool technology to create a comprehensive applicant profile that includes an assessment of work values. The complete applicant profile is then compared to a profile created by employers making job postings. Part of this process includes an employer work-values profile that helps capture the values of the organization in order to help ensure that users (both applicants and employers) are matched based on a mutual “fit.” Climber goes beyond the norm for this type of exercise by using sophisticated data analysis techniques and AI in order to help optimize the search/match process for its members. It is truly on the cutting edge when it comes to delivering candidate quality via tight searching/matching through the use of sophisticated data analysis techniques and AI.

Jobfox: Provides applicants with the opportunity to take a short personality assessment once they have registered. While the results of this assessment are not currently used to help match job seekers and employers, Jobfox members are able to see the results of their assessment and use them for self-understanding and developmental purposes. Jobfox also has a very nice skills matching technology in which applicants take the time to create a skills profile that is then matched against the skills profile that is created for each position that is posted by organizations using Jobfox. The result is a match score that provides candidates with an idea of which of the jobs available on Jobfox are a good fit for their skills and experience.

Monster: Monster has partnered with DDI, a leading assessment firm, to provide its clients with the ability to add assessments to any job posting. This product is known as the Monster Performance Assessment or MPA. The ability to provide the MPA for most job postings is made possible by DDI’s extensive experience with assessment tools as well as the data it has collected over the past several decades. The MPA package represents a groundbreaking offering, as it is the first time assessments have been sold transactionally as part of the process of posting job openings. The MPA does not use assessment to help candidates identify jobs for which they are matched; rather, it provides recruiters with much richer data about a candidate’s qualifications for the position of interest. This allows recruiters to quickly shortlist candidates based on a set of standardized, job related data. While there are still some details being ironed out, Monster’s approach represents an interesting and potentially effective way to help deliver quality. Monster is also the only one of the mega-boards that has been bold enough to step out with an assessment-related solution.

eBullpen: eBullpen is a job board that uses a personality assessment as part of the process used to match job seekers with available openings. eBullpen has been working on its process for several years now and provides a legally sound, useful approach. This approach asks companies posting jobs to outline the personality traits, activities, and requirements for the job. Job seekers provide data on their personality traits, work preferences, and qualifications. Matching alogrythms are then used to help match job seekers with available openings.

Each of these four companies are stepping out into uncharted territory and is doing so with a slightly different approach. The specifics of these approaches aside, the big picture is what matters the most here.

This big picture is the idea that each of these folks provide a way for job posters to break down a job into a series of key data points which are then matched to a series of related data points in a profile or assessment snapshot created by job seekers.

Detractors have often lobbied against this kind of matching, saying that no candidate (or company for that matter) is interested in taking the time required to create the profiles needed to fuel this kind of process. I firmly disagree. Good hiring takes an investment on the part of both the job seeker and the employer, and good investment often takes a bit of effort. The old adage, “you get what you paid for” along with my personal favorite “garbage in, garbage out” both apply to this situation. I am confident that as we travel further down the road of embedded assessment, the data will bear out the fact that the small amount of time taken to fuel the matching process will prove to be time well spent

Despite this, within the next five years I expect to see career portals offering an increasing amount of value by beginning to change the focus from quantity to quality, with assessments as one of the core elements of this switch.

I think all stakeholders involved in the hiring process, including applicants, stand to benefit from this trend.

Article and research by Charles Handler and Mark C. Healy

Over the past decade, there has been a significant increase in the number of assessment tools available for use in both hiring and developing employees at all levels of the organization.

By far the most common use of such tools has been for selecting hourly and technical workers. This makes sense, considering that the bulk of many organizations’ employees reside at this level and because success at many of these jobs usually boils down to a few key attributes such as:

• How fast can the person learn the job?
• Is the person going to steal, ditch work, etc?
• Does this person “get” customer service and will they help customers acquire what they need?
• Can this person communicate effectively?

As organizations gain more experience with measuring applicant qualifications, it has become common to find online assessment within the hiring process for hourly and skilled technical positions. Broader sales and marketing efforts by vendors and consultants have led to dramatically increased sales of these products. But what about managers and executives?

As we found in our latest survey of online prescreening and assessment trends, having candidates for supervisory roles (from 1st level all the way to the C-suite) complete questionnaires, tests, and simulations online is common, but there still seems to be a lack of assessment at these levels, especially when compared to the use of assessment for hourly jobs. Nonetheless, adoption of these tools has grown over the last 10 years and is likely to continue to do so over the next decade.

While the basic model for success with predictive assessments still holds, there are some key differences between successful assessment for managerial and executive-level positions:

• Managerial jobs require a slate of skills that are not often required for success at simpler jobs. These include things like leadership, business acumen, managing performance, strategic decision-making, conflict resolution, etc.
• A mistake at this level of hiring can be much more costly than for an hourly hire, mostly because managers are responsible for potentially hundreds of people and millions in both costs and revenue acquisition.
• Managerial hiring has a tighter integration with ongoing developmental strategy and activities as organizations look to maximize their investment in an individual by understanding and developing potential from within.
• There is a broad range of job complexity once you hit supervisory levels. The selection process for a front-line leader can differ vastly from that used for executive roles. Executives often complete day-long assessment centers while first-level supervisors are more likely to be presented with a simple in-basket or situational judgment exercise, which is scored automatically.
• Feedback becomes a more integral part of the assessment process. Pre-employment assessment for hourly jobs almost never provides candidates with any feedback at all.
• When making promotional or hiring decisions about managers, assessment requires a “whole is greater than the sum of its parts” approach. Trained assessors (often both external and internal members of the organization) make ratings based on a variety of data.

The above differences are clearly reflected in the past, present, and future of managerial-level hiring and development tools. Let’s start with the recent past.

The Past

Other than interviews, technical knowledge, and previous job experience, best-practice managerial assessment over the past 50 years has typically involved mimicry of the most important tasks faced by supervisors of both people and process.

Typical tools include:

• In-basket simulations. Also known as an “in-tray” outside of the U.S., these often-intense exercises imitate the administrative work of a manager in a fictional company. The traditional version of this tool presents a candidate with a stack of memos, a calendar, process descriptions, employee files, and other paperwork. The task is to respond to issues contained in the memos, with attention to the supplemental information about the company and its employees.
• Role-plays. Traditionally live and in-person, the candidate plays the role of a manager dealing with a poor-performing employee, dissatisfied customer, or other interpersonal situation. The interaction is scored (usually according to competency ratings) live or from a video or audio recording.
• Situational judgment. Usually in the form of multiple-choice questions, these assessments present a real-life work situation and ask the candidate to choose among alternative actions. Sometimes, multiple responses are correct, but each response is worth a different number of points.
• Business case/problem analysis. Some assessment systems require the potential manager to read over an operational or strategic problem and render an opinion or plan. Some include mathematical calculations and necessitate clear writing. Others essentially mimic a work-related intelligence test.
• Formal presentations. Particularly in the case of hiring at the director-level and above, and especially useful for selecting sales executives, the candidate is required to make a presentation about a product, strategy, or current issue. Aspects of the talk, including organization, clarity, and technical knowledge, are rated by assessors.
• Testing. There are several standard assessments that seem to be commonly used to provide insight into the personality and cognitive ability of managerial candidates. In many cases, the activities above get at the same basic constructs, but we have found that the higher the job, the more common it is to find testing as part of the assessment-center experience.

For higher-level jobs, the above ingredients are often combined into a one-half to three-day offsite often called an “assessment center.” Assessment centers are very expensive due to the need to have trained, qualified personnel to execute and score the exercises.

So, as the level of job gets lower, there is a decrease in the complexity of the assessment process.

The key factor uniting all of these diverse tools into an assessment center methodology is that they attempt to simulate managerial tasks and decisions in a real work environment.

Essentially a “test,” these highly regarded instruments ask a candidate to think and behave like a competent boss, instead of simply demonstrating a general trait (e.g., conscientiousness), skill (e.g., Intermediate Java), or ability (e.g., logical reasoning).

Moreover, these tools force the hand of more reliable, standardized data in a selection or promotional process, removing a worthwhile amount of subjectivity from key personnel decisions. There is even scientific evidence of decreased subjective bias on the part of hiring managers when compared with unstructured, ad-hoc interviews.

Another factor that underlies the use of managerial simulations is that they require some budget, effort, and dedication on the part of the HR and line organization, as does any worthwhile human capital initiative. Traditionally, these are expensive, somewhat cumbersome tools that are brought into a company by individuals who are just learning how to use them.

But simulations that realistically judge a candidate’s managerial prowess soon prove themselves to be invaluable to leaders making placement decisions.

Traditional managerial assessments have also demonstrated greater hiring accuracy. They’re more appealing to job candidates than personality inventories and job applications. They have a real relationship with future performance on the job when compared to other hiring tools. They are usually in the form of a statistical correlation with a measure of performance as a boss.

Decades of research in industrial/organizational psychology stands firmly behind these tools.

The Present

So what is the state of managerial hiring tools today? We contacted a wide range of assessment vendors to chat with them about the tools they offer for the selection of managerial to executive-level jobs.

Today’s market for these tools provides a continuum of options ranging from the old school to some thought-provoking new ideas. Here is a summary:

• Most assessment vendors and consultants understand that using the same content for lower-level jobs is not an acceptable strategy. Testing and measuring simple competencies does not account for the differences in what it takes to be a good entry-level worker and what it takes to be a leader/manager. We have seen an increasing number of vendors offering new products that leverage traditional approaches to making selection decisions for more complex jobs (i.e., assessment centers, in-baskets, etc.).
• Competency models, once thought of as a temporary fad, are now well-entrenched and help support linkages between employee selection and development. While competencies are all the rage for employee selection of all types, the link between selection and development tools for managerial-level and executive jobs is much stronger than it is for hourly jobs. Most managerial-level selection tools can also be used for development (with scores indicating strengths and areas to work on), and almost every provider offers a version tuned for both. This makes sense, as this kind of assessment is commonly used for identifying and grooming high-potential employees and for making internal promotional decisions.
• The in-basket is alive and well and living on the Internet. Most vendors offer an online in-basket that provides “assesses” with a realistic representation of a range of daily tasks required of someone at the appropriate job level. A simulation of an email in-box, coupled with organizational charts, employee histories, and process descriptions is most typical here, replacing the giant stack of memos with an MS Outlook-style in-box full of action items and hyperlinks supplemental information.
• Technology is being used mostly for delivery and administrative functions. There are many folks out there who are using the same basic content and format as has been used for decades. The biggest difference is that technology has been used to make the delivery of the content easier and to help make the tasks of scoring and report writing simpler as well.
• Online roleplay simulations are slowly starting to appear. We did see a decent number of products that provide the ability to use avatars and present assessees with more complex data and problems as part of the assessment experience. However, these are still well in the minority. Some of the technologies that will provide the true next generation of simulations do not yet exist. Forward progress in this area will be slow over the next few years.
• The more complex the job level, the higher the level of touch. Most of the standardized products we saw for the lower echelons of management use automatic scoring built into the product, whereas products for more complex and higher-level jobs use trained assessors to do scoring and feedback and act as role players.
• There is very little ability to customize content unless you have deep pockets. Most of the tools we saw did not easily support high levels of customization, though some firms sell versions of tools that differ by setting (e.g., healthcare, manufacturing, general office). However, there are different types of assessments for different types of jobs, and the tasks and competencies we saw are general for most leadership situations. As with assessment for individual contributors, you should analyze the job and carefully match assessment content to the actual work performed.
• There has been very little work done investigating the ROI of the online versions of traditional assessments. We still believe that asking someone to perform key aspects of a job is still the best way to evaluate how well they will do in their new role. Anecdotally, there are thousands who will testify to the impact of a good assessment program for selection and development of managers and executives.

The Future

So what does the future of managerial-level assessments hold? Here are a few ideas:

• More realism via simulations, particularly using avatars and online representation of complex, real-work environments.
• More artificial intelligence will be used in scoring, with technology that partially analyzes a written or spoken response by a candidate and, at the very least, speeds up the scoring and rating of completed materials.
• More data to support the value of managerial assessments.
• More diverse product offerings available as uptake increases in this hot area.
• Tighter ties with competency models will be created so that assessment and development will be linked more closely.

Conclusion

Today, best-practice managerial assessment involves a combination of the tried-and-true approaches of the past and the innovative media of the present.

For the organization that’s serious about leadership, managerial assessment tools (especially in-baskets and role plays) represent the key to truly separating those who aren’t qualified to influence one’s workforce from those who will aid revenue, retention, and overall performance.

We expect that as technology evolves, so will the automation of administrative- and scoring-related tasks and the level of realism associated with these assessments.

Article and research by Charles Handler and Mark C. Healy

For the last five years, Rocket-Hire has surveyed talent-management professionals who use Web-based screening and assessment products to enhance and quantify their hiring processes. Our goal for this research is to document trends in the usage of online screening and assessment tools in order to help provide a clear understanding of the popularity of these tools and their perceived positive and negative attributes.

Late last year, we again asked recruiters, HR staff, entrepreneurs, and hiring managers to tell us about their use and opinions of typical hiring instruments. This article highlights our major findings.

Today’s report is just a brief summary of the results of this study. For more data and an in-depth analysis of these trends, check out your May issue of the print publication, the Journal of Corporate Recruiting Leadership.

Overall, our findings from this year, and the trends that emerge over the five-year life span of this survey, continue to be positive. We had the largest number of respondents yet this year (141 recruitment and hiring professionals completed the survey, just a tad above last year’s 136).

The increase in use of screening and assessment tools and the positive benefits of these efforts continue to notate a strong trend toward their popularity and value.

On the negative side, results clearly indicate a continued lack of proper evaluation for these tools, which is even more discouraging when one considers that lack of understanding of the benefits of screening and assessment is still a primary reason for the failure to adopt them. Thus, the need to demonstrate the value of assessment by including evaluation as a core part of implementation is crystal clear and, in our opinion, remains the No. 1 challenge related to the use of assessment.

Now on to a more detailed review of our results.

Use of Applicant Tracking Systems

As with previous surveys, a majority of respondents employ an applicant tracking system. Specifically, 79% already have or will soon have one deployed, or are considering installation. As can be seen in the table below, no matter what the hiring demand, typical ERE readers are familiar with using an ATS.

Table 1. Use of Applicant Tracking Systems by Hiring Demand

Use of Prescreening Tools

About 57% use online prescreening to sort through their job applicants. We defined prescreening tools as:

Tools that gather information about, or ask candidates to respond to questions about their experience, skills, and qualifications in order to identify if they meet minimum job requirements. These tools are typically used early on in the staffing process.

The table below captures the usage rates of each of the most prevalent forms of screening.

Table 2. Usage rates of common prescreening tools

Common tools in use include the popular qualifications screen, technical skill certifications, and analyses of “fit” with company culture. Every year, it seems that more organizations are using prescreening as part of their online recruiting strategy. But the breadth of this practice varies, with some companies using these tools for all jobs while some use it for one or two alone.

Table 3. Extent of Prescreening Deployment

Use of Assessment Tools

In contrast to screening tools, online assessment tools were defined as:

Scientifically based screening tools that look more deeply into a candidate’s abilities, interests, and skills. These tools include personality measures, cognitive tests (i.e., verbal and quantitative skills), situational judgment tests, job simulations, etc. These tools are typically used for a more in-depth evaluation later on in the staffing process.

Underscoring the growing increase in the popularity of these tools, most common forms of online assessment showed an increase in rate of adoption over previous years, as comparisons with 2002 reveal.

Table 4. Usage Rates of Common Assessment Tools (2007 vs. 2002)

Specific skills and knowledge certifications and personality inventories are common assessment strategies once the pool of one’s qualified candidates has been reduced. Indicators of cultural “fit” and various cognitive ability measures are also widely used.

For another angle on the extent of assessment implementation, we asked respondents to indicate the level of jobs for which assessment is used for evaluating candidates.

Table 5. Job Level of Assessment Implementation by Job Level

2007 appears to be the first year that our data show a general pattern of use across all levels of an organization. Nonetheless, executive-level openings tend to entail more personal and direct methods of selection and placement, and were less likely to involve the use of any online tools.

Effectiveness of Screening and Assessment Tools

This year, 65% of prescreen users and 77% of assessment users felt their tools added value to their organization. In contrast, a full 21% of prescreen users and 10% of assessment users felt these tools did not add value. But this is difficult to judge, especially if one does not take the necessary steps to adequately answer the question. In this case, only 27% of users of prescreening tools and 36% of assessment users collected metrics to judge the quality of their interventions. Moreover, users of metrics tend to report success with prescreening and assessment.

Table 6. Perceived Effectiveness of Prescreening Tools

Table 7. Perceived Effectiveness of Assessment

Clearly, collecting metrics helps to answer questions about effectiveness and, interestingly, provide a positive response to these questions.

The Future of Online Screening and Assessment

Of those companies not currently using screening or assessment instruments, 42% feel they will implement one or more in the future. For them, Table 8 summarizes the sorts of tools under consideration.

Table 8. Screening and Assessment Tools under Consideration

As with the tools current users are seeking to use more of (or expanding to include in their hiring systems), qualifications screening, assessment of cultural fit, skill/knowledge assessments, and personality inventories dominate interest. Still, the mystery surrounding the value provided by these tools, as well as a prevalent intimidation factor, keeps companies from jumping in.

Obstacles to the Use of Online Screening

As usual, we asked users of online hiring technology (as well as non-users), what they perceived to be the biggest obstacle to the adoption of prescreening and assessment in their organization. Again, we received nearly identical responses to previous surveys.

Table 9. Single Biggest Perceived Obstacle to Deployment of Online Tools.

While 18% of users feel there are no obstacles at this time, many hirers feel that one or more roadblocks prevent prescreening and assessment from being integrated into the staffing process. These reasons mostly reflect a lack of knowledge of the benefits of more in-depth hiring techniques. Moreover, there appears to be a general lack of knowledge of safeguards and metrics designed to detect problems when candidate evaluation systems are in place, even among regular users of assessment.

But do these worries represent valid concerns, or is it simply fear of the unknown? Arguably, when assessment tools are properly matched to the demands of the open position, the only relevant concern is applicant identity (e.g., the stereotypical “smart older sister” taking a math test for an applicant) and test content security. But candidate enhancement of qualifications is common on resumes, in interviews, and for some personality traits/scales. Since many of the same concerns exist for paper-based processes, this should not limit any one organization from deploying Web-based hiring technology. Overall, most companies may legitimately cite few or no substantive reasons to limit their use of online recruitment and hiring techniques.

Summary

This year’s results let us know what’s really going on, and it confirmed what most of our previous surveys have found:

• Having an ATS installed is now nothing particularly innovative, especially if you work in a medium or large organization. For some, it’s hard to even imagine the old stacks of resumes and bulging file folders of applicant information.
• Prescreening and assessment are continuing their gradual penetration into the mainstream of recruitment and hiring.
• Qualifications screening, personality inventories, and skill and technical certifications continue to be the most popular online assessment tools. Assessments of cognitive abilities as well as fit with the company culture have expanded their footprint as well.
• Respondents report a lack of understanding, weak budgets, or a general lack of support for online tools as their primary obstacles to adoption or greater use of modern prescreening and assessment technology.
• Those organizations that formally evaluate their hiring practices tend to support the use of prescreening and assessment, but a large number of organizations are still failing to evaluate the effectiveness of their screening and assessment tools.

There clearly is a growing interest in scientifically derived hiring tools. This trend is encouraging but not particularly surprising; these numbers will continue to grow as more organizations understand the value in properly evaluating the impact of screening and assessment tools, seeing for themselves how quality hiring tools can improve a workforce.

As both authors have seen over a combined 25 years in screening and assessment consulting, skepticism about the value of these tools remains strong. We will continue to champion the use of evaluation to “close the loop” and reveal how more in-depth evaluation of candidates may positively impact the bottom line.

A word about our methodology: Respondents were evenly representative of recruiters, recruiting leaders, HR executives, business owners, and hiring managers, and represented a wide variety of organizations and hiring situations. Specifically, 59% were employed by organizations with 500 employees or more, with 34% working for a company with 5,000 or more employees. In contrast, about 20% were involved with recruiting for companies with 50 or fewer employees. In terms of employee acquisition activities, 34% hired more than 500 workers in a given year, but a majority of respondents (66%) made 500 or fewer hires per year.

Every year, I take a few minutes to reflect on the things I have noticed while working with both producers and consumers of screening and assessment tools. Overall, I am very encouraged by what I have been seeing. The market for screening and assessment tools continues to grow. This makes me extremely happy because we I/O psychologists know the value that is to be had via the use of quality assessment tools.

The science geek in me is also very happy to see strong investment in innovation. I am really pleased to see the ways in which quality content is being combined with technology to collect the mountains of data that are required to uncover underlying truths about the relationship between human traits and job performance.

One of the most valuable means of insight into assessment trends over the past five years has been the annual ERE/Rocket-Hire Screening and Assessment Usage Survey. If you have not taken the time to complete this survey yet, please consider taking five minutes to help us out. Your input really does make a difference, and we greatly appreciate your time. A link to the survey is provided at the end of this article.

The major theme for 2008 will be increased integration of assessment into carefully designed products. Much of the rationale behind these products is the reduction of the work required to implement assessment via a combination of data and technology that results in a turnkey product that is both relevant and easy to use. This trend is going to continue to increase the accessibility of assessment and lower barriers to entry that have long been limiting the use of assessments. A quick look at the trends outlined below clearly demonstrates support for this point. In 2008, I believe we will see the following trends continue to unfold:

1. Continued Interest and Use in Assessment. The market for assessment tools will continue to increase as it has every year for the past decade. This will be the result of a number of factors, many of which are discussed in more detail below. In my mind, however, the number one reason for increased adoption is still the fact that properly used screening and assessment is one of the most important ways to increase the quality of hiring decisions.
2. More Vendors to Choose From. As interest increases, so shall the number of options available to consumers. For the most part, this is a good thing, as the bulk of new companies and offerings are being created based on solid best practices. As always, there will continue to be less sophisticated or technically-sound options available. With so many options to choose from, extracting the most value out of assessment tools will continue to require a solid game plan and due diligence as the foundation of the vendor selection process.
3. Increase in Acquisitions for Content. As companies continue to work toward developing products and increasing use of assessment tools, many are discovering that it is easier to purchase content that has a strong history behind it than it is to create their own. This has also been playing out as existing assessment companies continue to purchase smaller companies in order to increase their product range. Recruitment Process Outsourcing (RPO) is also a huge driver of this trend. RPO will continue to be a major force in the future of hiring, and assessment should end up as a key part of the better RPO offerings.
4. Increased Integration and Packaging of Assessment into Off-the-Shelf Products. Directly related to the trends mentioned above, there has been a continued push towards the development of product suites that provide a hiring process into which assessment is embedded. The goal is to create a more turnkey product that can be lightly customized and quickly implemented. The more quality assessment that is baked into a well thought-out process that can be implemented relatively easily, the more quickly assessment can begin to return value. This equation also lessens the need for professional services and makes the product a bit more “foolproof.” The more turnkey the product, the less assurance one has that it is “dialed in” to a particular position or job. The decision to use a more turnkey product as opposed to a more customized approach should be treated as a business decision driven by the contingencies of one’s particular situation.
5. Increased Attention to Products/Sales into Vertical Markets. Again, as the trends noted above unfold, we are seeing an effort to package assessment products that have been developed specifically for certain vertical markets. This is driven partly by the fact that vendors are finding that it makes sense to promote and sell products in a more targeted manner. Health care is one of the hottest examples of this trend, but I have also noticed increases in product creation and promotion in manufacturing and financial services as well.
6. More Products Designed for the Middle Market. The labor-intensive nature of assessment tools has not traditionally lent itself to use by smaller or mid-sized companies. An increasing number of vendors are creating turnkey products designed to overcome the need for professional services and large-scale data collection exercises. As interest in assessment continues to grow, these products will be well positioned to provide value for a wider range of companies.
7. Movement into Job-Search Process. An increasing number of job boards/career portals are discovering that adding assessment can provide much better search/match functions than do traditional methods, such as keyword search. I have been pushing this idea for the past five years, but this year will provide a noticeable increase in the use of assessment tools as part of the DNA of many job boards.

While the above represent what I feel to be very positive trends, I also see the perpetuation of a few trends that I am not so wild about. These things include:

8. Continued Fragmentation. Most users of assessment seem to still take a test-focused viewpoint in which the emphasis is on the use of one specific test rather than on the result of a coherent process. While some situations warrant this type of approach, it is best to build a process in which assessment data is tightly integrated into other decision-making data. The good news is that all of the trends outlined above represent movement in the direction of less fragmentation.
9. Tactical Focus. Assessment is still being used in a highly tactical manner. It is most often considered as a way to fight fires, as opposed to a preventative measure. As consumers experience more success with the use of assessment tools, a more strategic focus will result. It will be some time before we see this type of usage occur, but we are headed in the right direction.
10. Continued Lack of Proper Evaluation. My head hurts from repeatedly banging it against the wall over this issue. Yes, it is hard to create the buy-in needed to properly evaluate the impact of assessment tools. However, without proper evaluation, it is very difficult to clearly demonstrate ROI and build a solid business case for the use of assessment tools. While vendors are trying to help design products that can assist in this area, proper evaluation requires commitment from the consumer. The consumer is the one who needs to collect the data relating to the business impact of assessment. While this is not an easy task, if we can put a man on the moon, we should be able to collect relevant performance data.

This promises to be a great year for screening and assessment. Continued interest by consumers has created a buzz amongst vendors who are, in turn, creating products designed to eliminate the headaches often associated with using assessment. Of course, there are always tradeoffs. The more turnkey a product is, the less it will be optimized for a particular local situation. The degree of customization required is a business decision that should be made based on a thorough needs and cost/benefit analysis. Some things never change!

Want to help us learn more about what is going on with the use of screening and assessment? Please take five minutes to complete our survey. We will be sharing the results with the ERE community this spring.

We I/O psychologists spend a lot of time talking to people about what they do, and we try to identify the human traits and experiences it takes to perform their jobs. We then use this information to develop components of the hiring process that will allow organizations to make predictions about which applicants will make successful hires. A big part of a successful hire often centers around hiring someone who will not leave the job in a manner that will cause a negative economic impact on his or her employer.

Of course, the good news is that in many situations, even a slight increase in the accuracy of predictions made during the hiring process can lead to huge levels of ROI. In my world, one of the most common reasons for using assessment and best practices is to reduce turnover primarily for this very reason. It takes effort to find, evaluate, hire, and train someone. All this takes two precious commodities: time and money. So, helping predict who will stay in a job long enough to offset these costs (at a minimum) and provide even more value through delivering good performance of job tasks, etc., is totally worth the investment.

Separation from a job (aka turnover) is one of the most frustrating but fascinating parts of the work I do. At the end of the day, even when we take great care to follow best practices and use good science, there is absolutely no way we can predict things like performance and tenure with high levels of accuracy. This is because we are dealing with humans, and each of us is as unique as a snowflake. Not only that, but each of us also brings to the table links to other people (such as family and friends) and a context that is broader and often more important than that of the workplace. This makes predicting anything about what humans will do a difficult endeavor.

The stuff I work with ultimately takes place in, and impacts, the real world. Because of this, I try really hard to pay attention to my daily experiences as a way to gain insight into the reality of what people face in their jobs. For me, these experiences help me put the things I work on for my clients into perspective and provide some really interesting nuggets that help me feel more in tune with the reality of people and their jobs, especially when it comes to a subject as interesting as turnover.

To illustrate this, I want to share an exchange I overheard between two cashiers while shopping at a major retail chain the other day, and to use this story to make a few points about the many variables that can make predicting things like turnover so hard. (Please keep in mind that the context here is retail associate-level jobs and, as such, this article applies mostly directly to this stratum of the workforce. However, the general ideas apply to almost any situation.)

The exchange went something like this:

Cashier 1 (African-American woman who appeared to be in her mid-20s): “How long have you been working here?”

Cashier 2 (Caucasian male who looked to be in his early 40s): “Just about five days. I was selling real estate before this, but the market is so soft right now that I needed to take another job to help save some money. I want to go back to school to become an appraiser, so I took this job while I save up. What about you? How long have you been working here?”

Cashier 1: “Only a few weeks. I was a barista for five years before this. I got sick of it though, so I decided to try something else. I really want to use my degree in African-American studies, but I can’t really find any work that will allow me to use it how I want. I am just going to work here until I can find the right opportunity to use it.”

Cashier 2: “Oh, that’s really interesting. I have been doing some work with the local cultural museum, and I know they are always looking for people to help out over there. You should check it out and see.”

Cashier 1: “Cool, that’s a good idea. I’ll be sure to check it out. Maybe I can quit this job and actually put my degree to work for me.”

I made my exit at this point, but I couldn’t help but stop to reflect on the meaning of that brief exchange in the context of what I do. My reflections included:

• Even though job duties are standardized and some work environments rely on workers having a core set of traits to carry out these duties well, everyone is still very different and comes to the table with a totally different history and context. This context often has more of an impact on tenure than anything else because it represents a reality that is highly personal and often transcends things like money and time.
• There are a lot of folks out there who may be seen as overqualified for some jobs but whose life-reality places them in situations where they end up needing the job to make ends meet. This can have both positive and negative impacts depending on the person, the environment, or even things such as the person’s relationship with management.
• The staggering number of options for doing something with oneself and the reality of the opportunities provided to those of us living in the United States often compels people of all ages and races (not just Gen Y or whatever other “Gen”) to look at jobs as temporary.

The above represent both good and bad things for employers. Individuality and personal context can have a huge impact on organizational performance that can be either positive or negative depending on how they are understood and managed.

The exchange between the two cashiers led me to the following thoughts, specifically regarding the realities of our current workforce, predictive hiring, and turnover:

• In many cases, hiring people for their raw abilities rather than for their experience or technical skills can be a good strategy, especially if one is having trouble getting butts in seats. Someone who is passionate about customer service and can relate to people can probably do well at jobs that require these things, even if they have no experience. This is one of the biggest value propositions for personality and work-values assessments.
• It is important to be honest about opportunities and what they offer to a job applicant. The reverse also applies; applicants should be encouraged to be honest about their goals for the job. Anything less is going to accelerate things like turnover.
• Organizations need to understand the demographic from which they hire. It is important to understand trends in how job seekers view employers and to do one’s best to adapt hiring practices to account for these things. This also applies to sourcing and even employment branding.
• Understanding of context should drive creativity. If the cold, hard reality is that a worker isn’t expected to stay at the same job long enough to make the hire pay off, seek creative ways to change this equation. This may include offering flex time, telecommuting, or benefits that are of value to the employee.
• Hire and/or train management to be understanding of the context that each worker brings to the table so that they can relate to that person and understand the value that his or her unique circumstances bring to the table.
• And, most importantly, tracking the relationship between data collected in the hiring process and objective, measurable outcomes is absolutely critical for understanding the economic impact of the reality that is our workforce. From this information springs the power to justify processes or to understand the need to adapt and change them.

So, the next time you are bored in the checkout line, do a little eavesdropping and stop to contemplate the bigger picture.

No facet of assessment has experienced as much change over the past decade as has the reporting of candidates’ results. Those of us who worked with assessment back in the dark ages can attest to the frustration experienced with the reporting of assessment results.

In those days the following were the norm:

• One would simply overlay a scoring key, enter in numbers into a worksheet, make some calculations, and arrive at a score for each applicant and perhaps a percentile rank based on an ancient set of norms.
• In some cases one could fax off an answer sheet and receive back a narrative report summarizing an individual’s performance.
• Examining aggregate data for more than one applicant required creating an Excel file or similar table by hand. These often had embedded formulas to help provide the information needed to make decisions about an applicant relative to other applicants and/or some set of job standards.

The commonality in the above activities was that no matter how it was done, reporting assessment results was a time-consuming process that yielded little insight about an applicant beyond a set of simple parameters that required users to do some extrapolating.

If one wanted to compare applicants head to head, creating a system for doing so was relatively complex and often yielded results that did not always inspire confidence in one’s decisions. Collecting aggregate data that spanned multiple steps in the hiring process to facilitate decision-making was pretty much out of the question.

Then again, back then someone in Peoria couldn’t instantly access home movies shot by someone in Mongolia either. Luckily for all of us, times have changed.

Reporting has become one of the strongest points of the assessment process and now provides users of with a set of powerful tools that add a tremendous amount of value to everyone involved in the hiring process. How does reporting add value? A quick look at the standards for modern assessment reporting broken down into several key areas should provide a clear answer to this question.

Area 1: Candidate Management Functionality

Those readers familiar with using an applicant tracking system have already had experience with candidate management. By candidate management I mean the basic information required to manage data from applicants for multiple requisitions. This may include a range of things such as assigning applicants to a job opening, sending out links to allow candidates to take an assessment, tracking those applicants through the stages of the application process, communicating with applicants, and pushing applicant information out to others involved in the hiring process.

The level of candidate management functionality provided varies quite a bit across assessment vendors. As the ATS and assessment worlds have continued to converge, the level of candidate management functionality has increased. This is especially true in products that have been designed for the mid-market where consumers may purchase one system that is intended to cover all the bases.

Most of the more evolved assessment providers offer some level of candidate management, and even the smaller ones are now starting to catch up in this area. This functionality is not always needed however, as many enterprise users rely on their ATS for this kind of thing. In these cases it is more common for assessment providers to pipe a few key pieces of data into the ATS system or in some cases to simply use both the ATS and the candidate management system in parallel.

Area 2: Dashboard Reporting

Dashboard reporting is a concept that is much bigger than HR applications. The concept is pretty simple: use a Web-based interface to pull in all of the data that is relevant for accomplishing a certain set of tasks.

Dashboard reporting has been the most significant change in the manner in which assessment is used. To be sure, the line between this type of reporting and candidate management is often blurred as both share some of the same key functionalities. The major difference is that the dashboard provides the ability to access much richer data about the applicant than does the candidate management system.

One of the biggest benefits of the dashboard is the ability to present information in layers that allow the user to focus only on relevant information. Below are the layers that are most commonly provided as part of dashboard reporting:

Level 1: Stack Ranked Comparisons

This usually takes the form of a screen that provides a stack ranked list of all applicants for a given requisition. Within this screen, the following data is usually provided for each applicant:

• Basic info such as name, date applied, contact info.
• Scores on each component of the selection process along with a pass-fail indicator, usually red, yellow, or green.
• An overall pass-fail indicator.
• Basic indicators of score on competencies critical to the job.

Level 2: Individual Competency Feedback and High Level Narrative Feedback

When one clicks on a piece of information on the Level 1 display, they are able to access more detailed information about each candidate. For instance, one may be able to view graphic representations of a candidate’s score for each critical competency associated with the job. This information may also contain some narrative to help decision-makers understand the ramifications of these competency scores in terms of key job requirements.

The goal of this level of information is to provide decision makers with extra data that can help them read in-between the lines when making decisions. This type of data is often displayed graphically using a format that displays an individual’s level of a given trait relative to normative standards or colors that demonstrate where an applicant stands on a certain trait relative to what is required for the job.

Level 3: In-Depth Information

Information at this level is usually accessed from the Level 2 display; however, there are often pathways in Level 1 that allow direct access to this information. This level of information usually contains the following elements:

• Detailed narrative. This involves very detailed narrative information about what assessment results may mean in terms of specific work behaviors. For instance narrative for someone who has displayed high level of teamwork based on assessment results narrative may read “Joe Candidate can be relied upon to defer to the needs of the team over his own personal wishes.” This type of narrative can get pretty in-depth as there are many positive and negative behaviors and tendencies associated with each of the competencies measured by an assessment process. This information can often get somewhat tedious and is not always going to be correct, since each person is an individual who may act differently in different situations. Over-reliance on this type of information is a common problem for those making hiring decisions.

In my opinion, the other types of level 3 information are of greater utility to decision makers. These include:

• Coaching recommendations. Based on the same type of data as are detailed narratives, coaching recommendations take things one step further by suggesting ways to help deal with the negative side of certain competencies. These type of recommendations are best suited to helping use assessment results for onboarding. I believe this is a very important way to help gain extra value from the assessment process.
• Structured interview questions. The generation of structured interview questions that are based on areas where more information is needed about where a candidate stands on a certain trait has become a market standard for assessment reporting. This type of function is an excellent way to gather more data about areas where assessment results do not provide the level of clarity needed for decision-making.

Area 3: Aggregate Data Reports

This includes any number of different reports that can be generated based on the raw data collected during the application process. The most common example are reports required by the EEOC, OFCCP, and other government agencies.

While many assessment vendors do have the bases covered when it comes to these major, mandatory reports, many also work with clients to create customized reports. These include things such as applicant flow, source of hire, and any other key metrics that are related to the hiring process.

Applicant tracking systems often cover some of these things, but there is an increasing need to use data from all parts of the hiring process, including assessment, to feed business intelligence. In many cases, vendors work with clients to set up customized reporting that will allow them insight into any number of important aspects of the hiring process.

Area 4: Flat Paper Reports

Of course, not every online hiring system actually offers all of the hi-tech reporting summarized in this article. In many cases the high-tech systems offer the ability to print an old-fashioned paper report that contains all of the same data summarized in the system. In many cases this type of report is very useful and is consistent with what hiring personnel have been using for years.

The type of reporting needed depends on a variety of variables. When looking at using an assessment vendor, your reporting needs are a key thing to consider.

Understanding what your hiring personnel need to get the job done as well as the key junctures in your hiring process will help provide some understanding about exactly what type of reporting you will need. Reporting is a usability issue and as such, end users of the reports should have some input when it comes to making decisions about reporting needs.

We I-O psychologist-types tend to be real data hounds. Much of the work we do for our employers/clients involves the use of data to investigate specific hypotheses in order to illuminate the underlying truth in a situation. The outcome of this work often has tremendous value to organizations because it provides them with hard data on which strategic decisions can be based. Additionally, the collection and analyses of data often helps us to identify new trends that we haven’t yet thought about.

Many of you who follow my articles know that I have a keen interest in the pre-employment assessment industry, and write quite a bit about its trends and happenings within. My interest in data and trends has led to an annual online screening and assessment usage survey.

The idea for this survey was born back in 2002, when I became frustrated over the lack of available information about the usage of pre-employment screening and assessment tools. This lack of information has been a challenge because though everyone seems to be saying that screening is becoming a hot area, there’s little actual data available to confirm this statement or to tell us how hot it really is. This lack of information also makes it hard for those of us who follow this industry closely to provide factual information about how companies are using online screening and assessment tools, and what the results of this usage have been.

At the end of this article, you’ll find a link to this year’s survey. Take a few minutes to help other members of our community by providing information about your company’s screening and assessment practices. The more data that’s collected the clearer existing and emerging trends will become. Last year, we had a record number of responses, a fact that seems to indicate the increased interest level in screening and assessment. Given the steady increase in interest and the lack of information about this industry, we feel the results will continue to have value for the ERE community. We look forward to reporting our findings right here on ERE sometime this coming spring.

In order to provide some extra motivation, here’s a quick summary of the trends identified in last year’s results (for a more in-depth look, please see last year’s results).

One hundred and thirty-six professionals completed the survey, representing an increase of 51% over the previous year. Respondents represented a wide range of company sizes.

Summary of Findings

• Most medium-sized and large organizations have adopted or will adopt an applicant tracking system, but the use of an applicant tracking system does not relate to perceptions of hiring effectiveness.
• Some hiring tools, like qualifications screening, skills testing/certification, and personality tests, are becoming standard features of a strategic hiring process. The use of some methods (e.g., online interviewing) is rare.
• Few companies formally assess their prescreening tools or use metrics of any kind as a decision-making aid. Those who do tend to be aware of the effectiveness of their hiring systems and perceive greater value in them.
• People professionals report a lack of understanding, weak budgets, or a general lack of support for online tools as their primary obstacles to adoption or greater use of modern prescreening and assessment technology.

Overall, the results of our fourth annual survey are the most encouraging yet. These results clearly show a growing interest in the use of assessment tools. Valid, Web-based approaches to recruiting, screening, and placing job candidates are fast becoming the norm at leading-edge, people-focused organizations.

This trend is encouraging and, in our experience, not surprising. We feel these numbers will continue to grow as more organizations understand the value in properly evaluating the impact of screening and assessment tools.

Unfortunately, there is still a very widespread failure to adopt the business intelligence mindset required to evaluate the effectiveness of screening and assessment tools. Primary obstacles include a lack of understanding of the value of these tools, as well as budgeting issues.

As long as this deficiency exists, it is not surprising that skepticism about the value of these tools remains strong. We will continue to champion the use of evaluation to “close the loop” and help ensure an increase in the understanding of how properly implemented screening and assessment tools can impact the bottom line.

This Year’s Survey

The goal of this year’s survey is to continue building on the information gathered from past surveys and to verify the fact that usage rates for online screening and assessment are increasing.

In order to help provide the ability to track changes in usage rates, this year’s survey is essentially the same as last year’s, with a few minor changes to help account for trends that have developed since last year. Specifically, this year we have added additional questions on budgeting and ROI for the use of screening and assessment tools.

While this is hardly a scientific survey, and its length limits the depth of the information that we are able to collect, there is tremendous value in the information your survey responses will provide. For instance:

• This survey will help to provide some reality to speculation regarding increases in the use of both online screening tools and online assessment tools.
• Your survey responses will provide information that is extremely useful for gaining a broader perspective on the evolution of the entire staffing process.
• The results of this survey will help provide companies with some basic benchmark information about how online screening and assessment are being deployed.
• The data from this survey will confirm our speculation that while many companies are using online screening and assessment tools, few are collecting the metrics needed to help them understand the true value they are providing. This is a problem that must be addressed, and we hope that verifying its existence will help to serve as a wake-up call.

All survey responses are confidential and individual responses will not be shared with anyone. By participating, you will help provide yourself and other members of the ERE community with up-to-date, accurate information about trends in the use of online screening and assessment tools.

We are interested in collecting information from anyone who is involved in the staffing process for his or her organization. This includes recruiters, hiring managers, staffing and HR executives, consultants, etc. We welcome participants from companies of all locations, sizes, and industries. We are not able to use information from individual vendors of screening and assessment tools or persons who may consult to multiple organizations.

Just click on “take the survey,” and you’ll be taken to the survey page. This page has all the information you will need to complete the survey. Please feel free to forward the survey to anyone you feel might be interested in participating. If you have any questions, just contact me at info@rocket-hire.com.

The results of the survey will be featured in an ERE article sometime this coming spring. We look forward to sharing our results with you.

One of the most difficult aspects of using assessment lies in gaining an understanding of the various pricing models associated with it. This understanding is critical when it comes time to budget for an assessment program. Even if your organization has experience using assessment, budgeting can be a very difficult issue.

Budgeting for assessment can be hard because of the following:

• Lack of understanding with assessment and its value proposition.
• Purse strings that are often owned by different groups within HR.
• General inexperience with assessment pricing models.
• Lack of understanding around price differences for off-the-shelf vs. customized assessments.
• The increasing commoditization of assessment via integration into other staffing tools.
• Lack of good benchmarking data.
• Extreme variation in assessment pricing.

The remainder of this article provides some basic information that I hope will shed a bit of light on assessment-related budgetary concerns. What this article does not do is provide specific benchmarking information that will allow you to magically understand how much you should budget for using assessment.

Consumers of assessment understand some general rules about assessment and pricing:

• The more customized the assessment content, the higher the up-front cost. Once up and running with customized assessment, however, the actual costs of testing often will not differ. The cost is in the investment required to create the customized content. Customized content often provides incrementally higher levels of ROI, so the extra investment is often worth it.
• The technology level offered by a vendor does not necessarily impact price. Many companies now offer assessment delivery platforms that are robust and have a variety of candidate-management features built in. These systems can be configured very quickly and easily. However, the more custom-technology-related work required to implement the system (i.e., applicant-tracking-system integration, customized reporting) the higher the price. These jumps can be steep when one starts getting into ATS integrations and the like.
• Just as the assessment market is extremely fragmented, assessment pricing is all over the map and price is not a reliable index of assessment quality.
• Assessment is getting cheaper as it becomes more integrated into the bigger picture as a value-add to staffing processes and the systems used to back them.
• At the same time prices are dropping, quality is actually increasing in many cases, due to the accelerated wisdom that is being gained via the ability to collect and analyze previously unthinkable amounts of data.

With the above in mind, there are three key areas of best practices related to the use of assessment impact pricing models.

Area 1: Setting Up the System

There are two key factors related to set up: doing the work required to understand key job requirements and technology system configuration.

The work needed to verify that the assessment content is job-related and the set up of the technology system are often inexorably linked, such that there is one “set up fee” that includes all the work needed to get the assessment process up and running. For example, using an off-the-shelf test from a vendor with a modern, robust candidate management system is getting to be less and less expensive. Set-up fees for this type of package usually range from $1,500 to $5,000.

As the complexity of the implementation goes up, these fees can often rise sharply. First is the level of technology integration required, with deeper implementation requiring much deeper pockets. Second is the level of due diligence related to understanding key job requirements.

If a full job analysis is required, you may pay north of $10,000, depending on the scope of the initiative. Conversely, using an off-the-shelf tool can be extremely cheap. Just remember that you get what you pay for and that taking the time to do a more in-depth job analysis can have tremendous value because it contributes to legal defensibility and also serves as a critical foundation for a variety of other HR activities.

Area 2: Assessing Applicants

This area involves charging for the test content or for the use of a system that collects a variety of different types of predictive data from applicants. There are a variety of different models related to test/system usage. Understanding them is critical when working on setting assessment budget.

The most common models include:

• Traditional-”test-centric” model. This is a flat per-head charge for any assessment used. Prices almost always follow a volume model such that the more units used, the lower the price. There is a huge range in per-test pricing with the cheapest tests now running at about $5 per head all the way to over $150 per head. The main variables are the number of tests and job level for which the assessment will be used. For instance, a test for entry-level retail workers will be much less expensive than a test for marketing managers. There is not always a direct correlation between price of an assessment and its quality, so be careful when considering this type of pricing.
• “Per-recommended” model. Because companies with more evolved technology systems often offer a multiple hurdle approach that involves the collection of data at various stages in the hiring process, many customers are wary of paying per click when there is high applicant volume and many applicants are screened out at the first step in the process. This has led many companies to create a variation of the per-head model in which charges occur only when candidates clear the first hurdle or for those who are judged as acceptable.
• Per-seat-license model. Some companies price their assessment based on the number of employees. This model is most common among vendors that sell packaged software systems and is often calculated to provide a price point for yearly use of the system
• Flat-fee-license model. Some companies offer a flat-fee model, an agreed-upon price for unlimited applicant evaluation. In many cases this model includes a threshold value such that over usage causes some additional fees.
• Pay-for-performance model. Some companies will offer a variation of any of the above models in which there is a service level agreement put into place regarding the level at which the use of assessment will return a return on investment. This model is rare but it is an excellent way to hold vendors accountable. It also serves as a quality filter when choosing assessment vendors because it takes confidence in one’s product and process to offer this kind of model.
• Combo model. It is common to encounter some combination of the above models. This is especially true as assessment becomes more tightly integrated into different types of human capital management systems and processes.

Area 3: Evaluating Assessment Accuracy/ROI

This involves the work done to “close the loop” and evaluate the effectiveness of the assessment used. Unfortunately, this area is often not a concern for many organizations because they do not take the time to do this type of work.

This may be one of the biggest mistakes when it comes to budgeting, because evaluating assessment ROI is the only way to demonstrate that the use of assessment is paying for itself.

Such demonstrations are critical for understanding budgeting and for providing the ability to obtain more budget for the continued usage of assessment.

There are two types of charges that relate to evaluating assessment:

• Configuring reporting. These costs can vary quite a bit, but most technology platforms have accounted for most of the major types of reports that one would need. This means that reporting costs are often not a big deal.
• Validation. This usually requires professional services to set up data collection, analyze data, and report findings. One can expect to pay between $10,000 and $25,000 or more for this work, depending on the depth of evaluation work required. While this may seem expensive, remember that validation pulls double duty because it provides legal defensibility and proof of ROI. Further, the increasing integration of technology into hiring is leading to a business intelligence mindset that is causing feedback loops to become increasingly automated and thus allowing for savings when it comes to professional services.

Conclusion

In general, the trend in assessment is moving toward commoditization. Prices are dropping because assessment is getting easier to use off the shelf and is becoming more tightly integrated into other staffing-related products such as job boards and applicant tracking systems. This is good news for many folks because it also supports availability of quality assessment to the middle market, where it was not previously affordable.

In days of yore, assessment came in two basic flavors: direct off-the-shelf or highly customized. In both cases, the revenue was related to the professional services required to ensure the assessment was properly configured, created, and evaluated.

In most cases, each local assessment implementation was customized and was costly due to the sheer volume of work that was required to set up, maintain, and evaluate the assessment process. The good news is that technology has allowed us to move toward models that involve fewer professional services while allowing for increasingly higher quality assessment content.

While reduction in professional services can be a good thing, just remember that you get what you pay for and that using a system directly off the shelf is likely to yield less-accurate results than creating an optimized assessment.

One of the other new trends that helps to bridge the gap between off the shelf and customized assessment is what I call “light customization.” This involves the use of standard “blocks” of content that can be stacked to help optimize the relevance of assessment without requiring the development of fully customized content.

I feel that this offers the best compromise between optimal configuration and quick implementation. This type of system is usually much cheaper than the old model for assessment while often delivering almost equal results.

The final answer of how much one should budget for the use of assessment is, “it depends.” I wish I could provide some hard-and-fast guidelines, but costs for assessments depend on a multitude of factors including how much customization you need; the level of technology required; the level of job you are assessing for; the applicant volume you plan to experience; and the level of follow-up evaluation you plan to use.

Do not skip out on the proper evaluation of assessment, even if it does add extra cost. Evaluation is critical because this is a central part of understanding ROI and provides the data needed to build a business case for the use of assessment tools.

Once you have implemented and evaluated an assessment process, you will have a much better starting point for the development of next year’s budget. Live and learn.

Recently, many of my clients have inquired about the viability of an assessment methodology known as employee profiling. As with many of the available assessment methods, there are legitimate concerns about the value of this method for helping organizations make legally sound and effective hiring decisions.

It is hard to provide a universal answer to this question, as most approaches out there are not technically incorrect or illegally, but rather, their degree of relevance and effectiveness depends entirely upon the situation in which a specific approach is to be used.

In order to provide some information and insight about the use of profiling, the remainder of this article highlights its pros and cons to help provide an understanding of why it may or may not be right for your organization.

What’s Profiling?

The profiling (or one-size-fits-all) assessment methodology is far from new, and there’s a wealth of evidence that demonstrates its effectiveness. Here’s a brief, generalized synopsis of how this type of assessment works:

• A set of high-performing employees is identified and they are given the opportunity to take an assessment designed to measure a variety of key characteristics that are related to job performance. In most cases, the assessment content used is the same no matter what the actual job is. It is the score patterns of respondents that provides the information that serves as a benchmark for hiring.
• The scores of high performers taking the assessment are used to create a benchmark score profile that defines a high-performing employee.
• The score patterns of applicants for the position for which the benchmark has been created are then compared to the benchmark profile during the hiring process. Those who most closely match the ideal profile are viewed as having the best chance of success and are recommended for hire in an effort to “clone” high performers.

This simple process has a variety of twists to it but at the end of the day, the underlying idea is the same. The idea is that the response pattern of high performers serves as a benchmark against which all applicants are filtered. The logic being that applicants that score the same as high performers have something in common which indicated that they too have what it takes to be a high performer.

With this methodology, in most cases, the same assessment content is used for a wide range of jobs. The differentiation defining the hiring profile is provided by the scoring template as opposed to differences in assessment content corresponding to differences in key elements of job performance.

Benefits and Drawbacks of This Methodology

As with any assessment method, profiling has both positive and negative aspects associated with it. Deciding whether it is the right methodology for your organization depends on a variety of factors.

Potential benefits of this method include:

• Intuitive. While many assessment methods can be a bit confusing, the idea behind profiling makes sense. Look at your best performers and develop a profile that can be used to make sure you hire persons who look just like they do.
• Fast. While many assessment methods can take a long time to implement, profiling can usually be implemented relatively quickly. One reason this method is so fast is that the same test content is usually used, so no content modifications are required. This is often the step that ends up slowing down assessment implementations.
• Cheaper. Profiling doesn’t require a job analysis or custom content development. Both of these things raise the cost of assessment implementations.
• Legally sound. While potentially not as legally sound as assessment methods using a job analysis, I am not aware of any legal problems that have arisen via the use of a profiling methodology. Remember that the litmus test for legal defensibility hinges on the job-relatedness of the assessment. Since profiling assessments often trade speed and ease of implementation for job relevance, there may be a slightly greater risk than with more traditional methods.
• Provides options to consumers. There are quite a few companies offering this type of method, so consumers are not limited to only one or two choices. Even better news is that many companies offer slightly different takes on how to implement the profiling method as well as different types of assessment content.

Potential drawbacks of this method include:

• Deficiency. Because there is only one assessment used for all situations, you may find that the content of this assessment does not fully capture all of the things required for performing the job in question. This can be an issue because if the test does not measure a key construct required for job performance, that piece of data will not be available as part of the candidate evaluation.
• Robustness of profile. In many cases, the profile created as the hiring standard is created by only a few individuals. The fewer the data points used to construct the profile, the less confidence one can have that it is actually a full and accurate picture of performance. When using the profiling method, have as many incumbents as possible complete the benchmark assessment.
• Failure to account for change. The profiling methodology does not account for the fact that the top performers surveyed may have had a different profile at the time of hire. In many cases, job performance and on-the-job training may allow an individual to learn and develop in many positive ways. Thus, the profile they provide may be an unrealistic one for persons who have not performed the job in question.
• Performance may be due to things not measured on test. One potential issue of the profiling methodology is that there is no way to actually prove that high performance on the test is actually due to things measured by assessment. This is especially relevant when one considers that most of these assessments use the same content for all jobs. This is one reason a job analysis is an important part of creating a selection system, as it allows one to ensure the key elements of job performance are accounted for in the selection process.
• Assumes inverse relationship. In many cases, bad performance is not simply the opposite of good performance. By hiring individuals based only on what high performers do, you may end up failing to account for some of the things that lead to low performance.
• Over-reliance on “the profile.” In many cases, there can be more than one profile that can define success. Holding too closely to one set of ideals may create unrealistic standards that can lead to over-reliance on some attributes and under-reliance on others. Hiring decisions should be the result of balanced information of many types, and the best hiring systems are designed to provide key decision-makers with a variety of information.

Many of the above criticisms can apply to other assessment methods. But they’re legitimate things to consider when evaluating the relevance of profiling methods for one?s own needs.

Here is a list of situations in which profiling is a good option for organizations to consider:

• An off-the-shelf assessment is needed quickly. Profiling is one of the fastest and easiest assessment methods to implement.
• The job in question is a relatively common one that does not have complex or unique demands. The fact that most profiling uses one set of content for all jobs means that the more mainstream the job, the more likely the profiling assessment content will be relevant.
• The organization feels strongly that there is one profile that is essential for performing a job. In many cases, a strong profile may have proven itself time and again. In this situation, profiling is more likely to be a good option.
• The organization is just getting started using assessment and is getting push back on more complex assessment methods. In many cases, the idea of conducting a job analysis and creating customized content is too advanced. The simpler and more intuitive profiling method may be an easier sell internally.
• The organization is too small or there are too few inc