Towards the end of 2003, I invited ERE readers to participate in my second annual “20 Questions About Online Screening and Assessment” survey. The goal of this year’s survey was to pick up where last year’s survey left off by continuing to identify important trends in the usage of online screening and assessment tools and by providing up-to-date data regarding usage rates for online screening and assessment tools. In order to help provide continuing insight into the development of major trends in the usage of online screening and assessment tools, this year’s survey asked many of the same questions as last year’s did. However, one major difference between this survey and the one conducted last year is that last year’s survey did not provide much in-depth information about the usage of applicant tracking systems and screening tools relative to that of assessment tools. In order to help collect this type of information, this year’s survey added a few specific questions related to the use of ATSs, separated questions referring to screening tools from those related to assessment tools, and provided questions designed to drill down more deeply into the characteristics of usage for both screening and assessment tools. First of all, a big hand to all of the ERE readers out there who were kind enough to spend some of their valuable time completing my survey. Thanks to you, I was able to collect a wealth of information. In fact, the survey provided so much good information that I will be presenting the results in two installments. So, without further ado, on to the results. Sampling Limitations Before I begin discussing my findings, I think it is important for me to bring up the fact that the conclusions reported in this article may have been influenced by several characteristics of the sample. These include:
Despite the possible sampling limitations, the results of this survey offer a lot of meaningful information that have allowed me to identify some interesting trends in the use of online screening technology. Sample Characteristics A total of 78 readers completed the survey (as compared to 65 last year). Of these, 62 were from the U.S., 2 were from New Zealand, and 1 was from Australia. The information in Figure 1 summarizes the relative percentages of the job titles held by survey respondents:
The data in Figure 1 indicate that the majority of respondents were either recruiters (36%), HR executives (19%), or managerial level-staffing personnel such as hiring, HR, or, staffing managers (27%). These results are very similar to those obtained last year. As it did last year, this data suggests that respondents represent a highly relevant sample population and should provide a very good source of information on the topics of interest. The information in Figure 2 summarizes the number of persons employed by survey respondents’ companies:
The data in Figure 2 indicate that 37% of respondents were employed by large companies, while the remaining respondents were fairly evenly split between small (24%), medium (23%) and medium/large (15%) size companies. These results were very consistent with those of last year’s survey. The information in Figure 3 summarizes the approximate number of hires per year made by respondents’ organizations:
The data in Figure 3 indicate a wide range responses in terms of number of hires made per year (from a low of 0 to a high of 16,000). Despite this range, the data indicates that the majority of respondents (61%) made between 0 and 150 hires per year. However, the percentage of respondents who indicate making between 500 and 20,000 hires per year was also rather high (30%). Surprisingly, very few respondents (9%) fell into the middle range (151-500 hires per year) in their hiring volume. Additional analyses comparing the relationship between company size and number of hires made per year confirm the expectation that the smallest companies in the sample make fewer hires, while the largest companies make the most hires. However, the hiring patterns of medium-sized companies (500-5000 employees) are less clear. Although, as one would expect, the biggest companies tended to make the most hiring decisions and the smallest companies the fewest, results still indicate a significant drop-off in hiring amongst mid-sized companies. I feel this can be partially explained by the slow economy during the time the survey was completed, most notably the huge slowdown in hiring that was experienced during 2003. The fact that such a high percentage of respondents (61%) indicated making less than 150 hires per year helps to confirm the fact that overall hiring volumes were low in 2003. The information in Figure 4 summarizes the length of time respondents’ companies have used the web to collect information from candidates for the purpose of making staffing and hiring decisions:
The data summarized in Figure 4 indicate that the majority of respondents’ organizations (56%) have been using the web as part of their hiring process for one to four years, while another 31% indicate having used the web for over five years. In the minority were organizations who either do not use the web for hiring or have done so for less than one year (13%). Additional analyses revealed no relationship between the size of a company and the time they have been using the web for hiring related purposes. Overall, the data indicate that the sample was relatively balanced in terms of respondents’ job titles and the size of their companies. Survey respondents represented companies of all sizes and the distribution of respondents amongst various sized organizations was relatively even. The fact that almost half of respondents are recruiters is complimented by the fact that about half of the sample hold managerial or executive-level positions. This suggests that the sample provides a good balance between respondents who are actually using online hiring technology on a daily basis and those who are making decisions about implementing such technology. Sample data also indicate that the majority of respondent’s organizations have significant experience using the web for making hiring decisions. Overall, the balance reflected in this sample means that the data represent an excellent cross-section of people who are likely to be using online screening. This suggests that the data summarized here offer excellent insight into high-level trends in the use of online screening and assessment technology. The fact that the sample characteristics are similar to last years’ should help to highlight ongoing trends in the usage of these tools. ATS Usage Survey results indicate that the majority of respondent’s companies (64%) currently use an ATS system, while another 8% indicate they are in the process of installing an ATS system. Additional analyses examining the usage of ATSs relative to company size revealed the following:
These analyses confirm the notion that larger companies are more likely to have an ATS or be in the process of installing one. The relatively high percentage of usage amongst small and mid-sized companies also provides information that companies of all sizes are using ATS systems. Additional analyses provide the following information regarding the relationship between a company’s ATS usage and number of hires they make per year:
Again, these data confirm the idea that companies making more hiring decisions are likely to have an ATS. They also reinforce the notion that even companies with medium and small hiring volumes (i.e., smaller or medium sized companies) are still likely to be users of ATS systems. Survey data also indicate a very interesting relationship between companies that use ATSs and the percentage of successful hires made by respondents’ organizations over the past two years. For example, amongst those with ATS systems:
These results are very interesting because they clearly indicate that the absence of a relationship between the use of an ATS and the perceptions of the success of hires made using the system. These results reinforce my strong opinion that, while ATSs are extremely useful, an ATS alone is not sufficient for increasing the overall quality of hiring decisions. Overall, these results confirm the fact that the adoption curve for ATSs has reached maturity and that almost all medium to large sized organizations are using some form of an ATS. Another interesting point is the fact that even small organizations are using ATS systems. This widespread usage of ATSs by companies of all sizes suggests that there are ATSs available to suit a wide variety of hiring needs. Of additional interest is the fact that there seems to be little relationship between the usage of an ATS and the actual quality of hires made using the system. This confirms my opinion that, while ATSs are extremely useful for management of candidate data, they do very little to help organizations predict which candidates will succeed. Current Usage of Online Screening Tools As with last year’s survey, one of the major goals of this survey was to collect data about usage rates and characteristics for online screening and assessment tools. Unlike last year’s survey, this year I split screening and assessment into two separate sections. I did this in order to help us drill down a bit more deeply into these two somewhat distinct types of online staffing tools. This section deals with the usage of online screening tools (online assessment tools will be addressed next month in Part 2 of our results). In the survey we defined these tools as: “Tools that gather information about, or ask candidates to respond to, questions about their experience, skills, and qualifications in order to identify if they meet minimum job requirements. These tools are typically used early on in the staffing process.” This section of our survey was designed to gather the information needed to answer the following general questions about online screening:
The data summarized in this section provides some interesting answers to these questions. For instance, 59% of respondents indicated that they use some form of automated screening tools. Table 1 summarizes the usage rates for 4 specific types of tools: Table 1: Type of Screening Tools Used
| Type of Screening | % Currently Using |
| Automated qualifications screening | 50% |
| Resume scanning tools | 28% |
| Biodata, personality questions | 14% |
| Index of “Job Fit” | 26% |
This information suggests that qualifications screening was the most popular form of screening currently in use. The usage rates for each of these tools are very consistent with last years’ results, indicating a stability in their usage rates. Effectiveness of Screening Tools The survey also sought to gather information regarding the perceived effectiveness of screening tools and the usage of metrics to help evaluate their effectiveness. These survey questions also provided some interesting results. For instance:
Additional analyses examining perceptions of screening effectiveness amongst respondents indicating that their organizations DO collect metrics provided some very interesting information. These analyses revealed that:
This information suggests that organizations that take the time to collect metrics have a better understanding of the effectiveness of their screening tools and seem more likely to use tools effectively. This information may seem like a no-brainer, but it clearly reinforces the importance of collecting metrics related to the usage of screening tools. Without hard data, it is much harder to know how effective your screening tools are. In my experience it is much more difficult to convince the powers-that-be to allocate more money for screening tools when you lack the ability to demonstrate the ability of these tools to provide ROI. The information in Table 2 summarizes respondents’ opinions regarding the effectiveness of screening tools currently used by their organization: Table 2: Perceived Effectiveness of Various Types of Screening Tools
| Type of Screening | % indicating this form of Screening is effective for their organization |
| Resume scanning tools | 55% |
| Automated qualifications screening | 65% |
| Index of “Job Fit” | 75% |
| Biodata, personality questions | 82% |
The information in Table 2 suggests that the tools that collect more in-depth information from applicants are more likely to be seen as being effective. I do not think this is a coincidence. Tools that examine applicant characteristics such as personality and biographical history should do a much better job of predicting applicant success then tools that collect only cursory, surface information (i.e., resumes). It will be interesting to see the relationship between perceptions of effectiveness of these screening tools versus that of assessment tools (this information will be provided in Part 2). Finally, respondents were asked to indicate the single biggest problem they have experienced related to the use of screening tools. The most common responses to this question included:
Again, these responses are not surprising to me. I have been hearing these same reasons for several years now. While I agree that screening tools are time consuming and can be hard to set up, I also understand the many benefits of using these tools. As with anything else in life, it is very hard to get something for nothing and screening is no exception. An investment in overcoming the difficulties with setting up screening questions correctly and positioning the value of screening to applicants will pay off handsomely in the end. Conclusions I feel that the survey results presented here provide a very good reflection of the major hiring-related trends we all lived with over the past few years. These include:
Stay tuned, Part 2 will discuss results related to the current usage of assessment tools, and future trends in the usage of both screening and assessment tools.