2005 Online Screening and Assessment Survey Results, Part 2

article by Mark Healy & Charles Handler

Organizations have been deploying pre-employment tests and assessments on computers for at least two decades. Originally just pencil-and-paper tests adapted to a computer interface, modern hiring and employee development tools have surmounted difficulties with data capture and reporting, user access, and database management to bring a new level of technical sophistication to recruitment and hiring programs. Now, six years into the 21st century, human resources and recruiting departments are keenly focused on the dispersal of postings, applications, and assessment tools to job applicants via the Internet. This revolution in the way companies recruit and hire people is moving quickly. So for the third straight year, I’ve asked users of online screening and assessment technology about their current and future use of web-based hiring systems.

This year we were able to collect data from 90 hiring professionals. The data offered us an excellent snapshot of current organizational strategies, concerns, and insights around the use of technology based screening and assessment tools.

In Part 1 of this article series on our findings, we detailed current usage rates and feelings about screening systems such as online job applications, applicant tracking systems (ATS), qualifications screening, and resume scanning. Moreover, we investigated trends in tracking the effectiveness of these tools, and how organizations could do a better job of understanding the systems they have in place. Here, in Part 2, we dive deeper into automated hiring processes by considering the current and future use of online assessment tools.

Assessment vs. Screening

Although definitions vary, the term “assessment” has been distinguished from “screening” in today’s human resource environment. For the purposes of our survey — and in alignment with this trend — we defined online assessment tools as: “Scientifically-based screening tools that look more deeply into a candidate’s abilities, interests, and skills. These tools include personality measures, cognitive tests (i.e., verbal and quantitative skills), situational judgment tests, job simulations, etc. These tools are typically used for a more in-depth evaluation later on in the staffing process.” In other words, assessment tools evaluate job candidates much more thoroughly than prescreening devices, and are often used once an initial hurdle or multiple hurdles have been successfully passed by applicants. Assessment tools tend to evaluate the level or quality of a skills and abilities that are essential for job performance rather than the extent to which an applicant’s experience qualifies them to perform a specific position. As in past years, our survey revealed some very interesting information about the use of assessment tools. The remainder of this article provides a summary of these results.

The Use of Assessment

Of our 90 respondents, 68 companies (76%) utilize assessment tools — either online or in traditional paper and pencil format. Of these, 50% deploy one or more tools online, either in a proctored environment or over the Internet for access from any web-enabled computer. Organizations surveyed varied quite a bit by size, but this — and number of hires per year — had no bearing on whether or not a company used assessment tools. This is in contrast to last year’s sample, when organizations with 5,000 or more employees used assessment at much greater rate than smaller companies. The ubiquity of applicant tracking systems has engendered a slight increase in the rate of companies who integrate assessments into these common HR platforms. Specifically, 22% of companies using assessment integrate candidate results into their ATS, compared with 13% last year. This generally low rate of assimilation (under 25%), however, is not really surprising; although use of an ATS has become standard in most large HR operations, the full capabilities of such systems are not typically utilized. The breadth of assessment use throughout an organization varied quite a bit across the study as well.

As can be seen in Table 1, the typical organization deploys assessment tools as part of the hiring strategy for only a subset of jobs. Only 19% utilize online assessment for all jobs worldwide, and an additional 20% use assessment for all domestic jobs. These results are similar to last year’s findings, where 37% of organizations used assessment either world- or country-wide.

Table 1. How is online assessment deployed in the organization?

All jobs within a business unit, but not all business units 4%
All domestic jobs 20%
All worldwide jobs 19%
Specific local jobs only 41%
Not sure 17%

For another angle on the extent of assessment implementation, we asked respondents to indicate the level of jobs for which assessment is used for evaluating candidates.

Table 2. Use of online assessment at different job levels.

Entry level/hourly 46%
Lower level management/professional 49%
Middle level management/professional 43%
Higher level management (director) 37%
Executive/vice president 26%
Other 7%

The results of this year’s survey suggest that this is one aspect of online assessment that has evolved. The in-depth evaluation and comparison of pre-qualified candidates using online assessment tools is no longer relegated to entry-level positions or technical professionals. At only the senior leader level does use remain under 30%; this is not unexpected, given the one-on-one, grassroots nature of executive recruiting and promotional systems. Unlike job level, rates of use for different types of job settings appear to vary considerably. Table 3 reveals these distinct trends.

Table 3. Use of online assessment by job setting.

Customer service 41%
Manufacturing/labor 12%
Skilled trades 35%
Account management 34%
Call centers 35%
Managerial/supervisory 46%
Administrative 35%
Information technology 43%
Retail 12%
Consulting/advising 19%
Sales 35%
Professional 46%
Other 26%

These results reflect not only the sample of survey respondents, but a certain level of maturity in the development and implementation of assessment in the early 21st century. For example, use of online assessment tools for managerial, IT, and customer service hiring is common in part because so many different cognitive ability tests, personality inventories, and other in-depth assessments have been developed specifically for these sorts of jobs. Moreover, validated assessments of key skills and competencies required for success in these roles have been available for years in paper-and-pencil format. Not insignificantly, organizations tend to imitate others in their industry, so a bit of a “snowball” effect is reflected here as well. Completing an online assessment has become relatively standard for customer service, managerial, professional, and IT jobs, but manufacturing and retail hiring situations are still commonly handled via face-to-face evaluation of local, “walk-in” applicants. Online assessment of these candidates is still in its novel phase of evolution.

Types of Assessment Tools in Use

The rates of usage of various types of tools by organizations employing an assessment strategy provide insight into the growth of this area, especially when compared to our previous surveys. Although this year’s sample of people professionals represents a largely different group of individuals compared to last year, the substantial increase in the adoption of most types of tools is difficult to put aside. Even more telling, Table 4 displays the wide disparities among the rates of use of different types of assessments.

Table 4. Rates of adoption of common assessment tools.

Type of assessment 2005 2003-04 2002
Personality measures 34% 30% 21%
Assessment of “fit” with company 35% 27% 29%
Cognitive ability 46% 27% 26%
Biodata 15% 10% 14%
Skills/knowledge 53% 40% 12%
Background investigations 54% 30% 31%
Simulations 18% 14% 10%
Online interviews 15% 4% 19%

In line with previous surveys, assessments of background, specific skills and knowledge, and cognitive ability are becoming routine. However, this year’s results signify a substantial jump in the level of adoption of these instruments. Biodata inventories, online interviews, and simulations continue to be less often used. Moreover, the most common types of assessments realized the largest gains in usage — particularly skills/knowledge assessments, background investigations, and cognitive ability tests.

Effectiveness of Assessment Tools

With the wider implementation of these tools, are companies finally realizing the value of structured, validated assessments to their people strategy? Table 5 displays overall responses to, arguably, the most important question surrounding the implementation of any people strategy or tactic. As can be seen below, whether an organization actually attempts to answer this question is critical to the eventual answer.

Table 5. Perceived effectiveness of assessment vs. collection of metrics.

Does assessment have a positive impact on your organization? Yes No Not sure
Overall 70% 7% 22%
Organizations collecting metrics 88% 0% 12%
Organizations who do not collect metrics 64% 16% 20%

As with previous years, and with just about any people program or strategy, less than one-third (specifically 28%) of the companies who use assessment formally collected metrics to determine if their assessment strategy adds value to their organization. Of those that do, there seems to be strong confirmation of the value of assessment. Those who do not, in some cases, have determined that assessment does not positively impact their company, even if they have not directly measured its effect. These results match those for online screening tools, as detailed in Part 1 of this piece. Since the collection of metrics does seem to contribute to perceptions of value to the organization, we also asked if the collection and presentation of metrics help to make a business case for the continued use or expansion of assessment tools. According to Table 6, the majority of companies collecting metrics are able to use them to build a business case for the use of assessment.

Article Continues Below
Table 6. Do metrics help build a business case for assessment?

Yes 71%
No 11%
Not sure 18%

An evaluation of a company’s hiring system — or just parts of it — may involve a variety of different metrics and measures that consider “success” from a variety of angles. We asked survey respondents to indicate the metrics they use to evaluate the impact of their assessment systems. Typical measures include:

  • Comparisons of assessment scores versus tenure and turnover
  • Traditional validation studies, including statistical analysis of the relationship between prescreen and assessment scores and job performance measures
  • Adverse impact and diversity indicators
  • Standard HR metrics, such as time-to-fill position and cost-per-hire
  • Formal and informal impressions and opinions of managers, HR and recruiting staff, and job candidates

These vary in terms of effort required and the time span required for data collection. Still, these strategies may be systematized and applied to all hiring strategies in the organization, and people professionals can easily build expertise in this area as use of metrics permeate their hiring processes.

The Future of Online Screening and Assessment

Despite the expansion and pervasiveness of online screening and assessment, many organizations have not yet begun their implementations, or even their purchase, of these tools. However, of those companies not currently using screening or assessment instruments, nearly three-quarters of them (73%) feel their organization will adopt them in the future. For these potential adopters of online hiring technology, Table 7 summarizes the sorts of tools under consideration.

Table 7. Screening and assessment tools under consideration.

Type of screening/assessment Percent considering use
Resume scanning tools 26%
Qualifications (experience, education, etc.) 46%
Personality measures 23%
Assessment of “fit” with company 34%
Cognitive ability 17%
Biodata 11%
Skills/knowledge assessment 60%
Background investigations 34%
Simulations 17%
Online interviews 23%
Don’t know 6%

Consistent with previous surveys, and with the large market space filled by the many competitors for your business, qualifications screening, assessment of cultural fit, skill/knowledge assessments, and background investigations dominate the interest of companies seeking to dip their toe into the ocean of online hiring tools. Companies will consider, compare, and purchase these systems using a variety of techniques. Table 8 details these methods for selecting recruitment and hiring technology.

Table 8. Method of comparing and purchasing assessment tools.

Shopping and purchasing method % using method
Formal RFP process 26%
Informal decision making 18%
Recommendation from consultant 6%
Via partnerships/services from existing vendors 16%
Don’t know 26%
Other 8%

Clearly, few trends emerge, with formal requests for proposal (RFP) the most popular — but by no means majority — decision-making tool. So what is keeping many organizations from adopting online screening and assessment tools or expanding their role in recruiting and hiring?

Obstacles to the Use of Online Screening

Greater usage of online prescreening and assessment would occur if it wasn’t for the obstacles and limitations perceived by organizational leadership and people professionals. These anxieties ó whether grounded in reality or not — present real barriers to the growth and enhancement of hiring success in every type of organization. Presented in order of endorsement, Table 9 highlights the concerns held by organizations seeking to adopt hiring technology.

Table 9. Percentage of respondents noting concerns and obstacles to the adoption of online hiring tools

Too costly; lack of budget resources 27%
Lack of knowledge in organization 26%
Skepticism about the ability of screening to provide results 22%
Decision makers do not believe it is worth the cost 17%
Hesitation due to legal issues 12%
Hesitation due to security issues 11%
Tools will negatively impact the candidate experience 11%
No obstacles 9%
Technology is still too new 9%
HR not interested in innovation; reluctance to change 8%

Another way to consider these feelings is to look at the “single biggest obstacle” perceived by survey respondents. For the most part, the data in Table 10 reflect the results in the previous table.

Table 10. The single biggest obstacle to the adoption of prescreening and assessment?

Too costly; lack of budget resources 19%
Skepticism about the ability of screening to provide results 16%
No obstacles 14%
Tools will negatively impact the candidate experience 11%
Technology is still too new 11%
Decision makers do not believe it is worth the cost 7%
Lack of knowledge in organization 7%
HR not interested in innovation; reluctance to change 7%
Hesitation due to security issues 5%
Hesitation due to legal issues 4%

In general, these concerns center around:

  1. The value of spending money and time on more advanced assessment
  2. Candidate reactions to hiring tools
  3. Lack of knowledge regarding the positive implications of assessment tools

With 14% indicating no obstacles, are these legitimate concerns or merely anxieties experienced by inexperienced, paranoid, or jaded decision makers? To answer that question and others, we will, in Part 3 of this article series, delve into a deeper analysis of the overall findings and general implications of these results.

Dr. Charles Handler is a thought leader, analyst, and practitioner in the talent assessment and human capital space. Throughout his career Dr. Handler has specialized in developing effective, legally defensible employee selection systems. 

Since 2001 Dr. Handler has served as the president and founder of Rocket-Hire, a vendor neutral consultancy dedicated to creating and driving innovation in talent assessment.  Dr. Handler has helped companies such as Intuit, Wells Fargo, KPMG, Scotia Bank, Hilton Worldwide, and Humana to design, implement, and measure impactful employee selection processes.

Through his prolific writing for media outlets such as ERE.net, his work as a pre-hire assessment analyst for Bersin by Deloitte, and worldwide public speaking, Dr. Handler is a highly visible futurist and evangelist for the talent assessment space. Throughout his career, Dr. Handler has been on the forefront of innovation in the talent assessment space, applying his sound foundation in psychometrics to helping drive innovation in assessments through the use of gaming, social media, big data, and other advanced technologies.

Dr. Handler holds a M.S. and Ph.D. in Industrial/Organizational Psychology from Louisiana State University.

LinkedIn: https://www.linkedin.com/in/drcharleshandler