Receive daily articles & headlines each day in your inbox with your free ERE Daily Subscription.

Not logged in. [log in or register]

Results from the 6th Annual Rocket-Hire Online Assessment Usage Survey

by Aug 3, 2009, 11:20 am ET

Article and research by Charles Handler and Mark C. Healy

For the last seven years, Rocket-Hire has surveyed users of web-based pre-employment assessment tools, so we again asked members of the ERE community to tell us about their usage of typical pre-employment screening, testing, and assessment programs. As with years past, we zeroed in on the pulse of pre-employment assessment usage. And in an increasing climate of legal scrutiny for testing, and the hoopla surrounding the Ricci case, we decided to focus the content of today’s article on two issues that are inexorably linked: Implications of evaluating one’s assessment strategy, and attention to relevant legal issues.

Those interested in obtaining a copy of our full report can email us (chandler@rocket-hire.com) and we will be sure to send you a full copy once it has been completed.Or, check out an upcoming Journal of Corporate Recruiting Leadership, probably the October 2009 issue, where we’ll have an in-depth analysis.

A Word About our Methodology
This year, 148 recruitment and hiring professionals completed the Rocket-Hire Online Assessment Usage Survey. Respondents were evenly representative of recruiters, recruiting leaders, HR executives, business owners, and hiring managers, and featured a wide variety of organizations and hiring situations.

Use of Assessment Tools
Overall usage of assessment tools was generally about the same as in past years — roughly two-thirds of respondents. Of that two-thirds, 54% are deploying both paper-based and online assessment, and 30% are using exclusively online assessment. The remaining employ only paper-based tools. Most use a variety of different assessment methods, with the majority using between one and three different types. The table below reveals the proportion of firms using various common tools.

Usage rates of common assessment tools

Skills and knowledge certifications and personality assessments continue to be popular, with indicators of cultural “fit” and various cognitive ability measures also widely used. In addition, we observed a surge in the adoption of web-based background investigation strategies.

Interestingly, 2009 highlights a trend emerging over the last few years: It is now just as common to find organizations deploying assessment company-wide as it is to find it used for only one or two jobs. These findings reflect the fragmented use of assessment that we have experienced in our client work. To be sure, there are tons of ways assessment can be used, but there is no one major trend in how or how much it is implemented.

Effectiveness of Screening and Assessment Tools

This year, 80% of assessment users felt that assessment had a “positive impact” on their organization. In contrast, only 3% of assessment users felt these tools did not add value. However, only 64% of assessment users collected metrics to judge the quality of their interventions, and 79% judged ROI using one or many indicators. However, definitions of “ROI” vary widely, with some approaches considering a broad, cursory opinion from managers, whereas only a few use a true scientifically derived method such as a validation study.

Wide variation in the real-world evaluation of ROI clearly reveals that, while many companies do collect metrics, few of them are going about this in a way that reflects best practices. This is cause for concern given the fact that these same best practices create the foundation for demonstrating the legal defensibility of assessment measures.

Legal Issues

This year’s Supreme Court decision in favor of firefighters who challenged the City of New Haven, Connecticut, has once again focused attention on the legal aspects of pre-employment assessment. In anticipation of the High Court hearing this case, we wondered how much typical users paid attention to legal and regulatory issues related to using assessment to evaluate candidates.

Percent indicating legal issues are “central to this activity” or “very prominent”

These results express both positive and negative connotations. On the positive side, it seems that a good number of companies pay attention to legal issues related to testing and assessment. On the other hand, perhaps more firms should consider devoting resources to demonstrating and documenting legal defensibility. The lack of understanding of what is actually required for ensuring compliance and defensibility may create problems as both assessment usage and legal scrutiny increase.

The energy spent on legal considerations does not relate to the extent of assessment deployed, or size of the organization, nor number of hires. As with overall usage of assessments, legal concerns do not follow any pattern across survey respondent demographics or assessment strategies.

Legal defensibility is not just another set of meaningless hoops that must be jumped through and checked off. Proper evaluation of ROI via validation and legal defensibility are actually two sides of the same coin, the common goal being to document the job-relatedness of all tools and processes used to select employees.

Though the continued economic turmoil has put a damper on hiring, the adoption of in-depth assessment tools continues to grow, especially as more organizations devote resources to find out if testing and assessment is worth it. The bottom line: Quality hiring tools can improve performance and employee engagement (i.e., provide ROI) while also ensuring the legal defensibility of hiring practices, but it is essential to demonstrate how much they actually benefit your organization.

This article is provided for informational purposes only and is not intended to offer specific legal advice. You should consult your legal counsel regarding any threatened or pending litigation.

  • http://www.mdaleadership.com Amy McKee

    What’s the desire for companies to 1) use web-based simulations with video clips or avatars to show a vignette, and then have candidates choose the most appropriate response (typically in multiple choice format)vs. 2) have a phone-based simulation presenting a live situation in which the candidate must interact with a role player/assessor? If the costs were not too dissimilar (say 300 vs. 500), would companies find value in the live simulation to pay more?

  • charles handler

    Good question Amy. I think that the best answer is, it depends… Both options have value depending on the situation at hand. Companies make use of both of these tools as a part of their hiring process. The use of web based simulations is just getting started but you should expect to see more and more options for this kind of assessment in the near future. These assessments are an excellent way to place candidates in realistic job situations and can be way more engaging then traditional tests. Phone based simulations are less common and I think their use is moving in the opposite direction. The exception to this would be a behavioral interview, which functions somewhat like a simulation as the candidate is asked to discuss how they would handle various scenarios that are relevant to the job. Phone based assessments are not automated so their cost is likely to be higher due to the fact that each phone call requires more resources. On the other hand, there is more richness of information from a one on one dialogue then from a more static simulation.

    When it comes to pure simulations, not including interviews, I think the trend will be an increase in the use of web based simulations and fewer phone based simulations. This makes the most sense from an economic perspective.

  • Pingback: It’s Time for Rocket Hire’s Annual Online Assessment Usage Survey - ERE.net

  • Pingback: Candidate Experience – How Candidates Want to Learn About Job Content, Part 5 of 6 - CollegeRecruiter.com

  • Pingback: The Assessment Time Capsule: What Has and Has Not Changed in the World of Pre-employment Assessment - ERE.net

  • Pingback: The 6th Annual Pre-Employment Test Usage Survey by Rocket-Hire

  • Pingback: Validation: Overcoming Inertia to Prove the Value of Staffing as a Profit Center - ERE.net