Screening Foundations: Building an Effective Foundation Online

This article is the second of a two-part series focusing on building effective foundations for online screening. Last month’s article provided some practical tips for helping recruiters to identify essential job requirements in the absence of a formal foundation-building process. This month’s focuses on the foundation-building tools that accompany online screening systems. What Does a Good Foundation Look Like? Understanding the issues discussed in this article requires a quick look at the ingredients that make up a good foundation. (For a more thorough review of the principles of foundation building, please take a look at Part 1 as well as my earlier article on the subject). First of all, I want to clearly define what I mean by “foundation.” To do this I like to use the analogy that screening foundations are like blueprints for job success. They provide a clearly defined picture of exactly what is required for effective performance at a given position. This information then dictates the screening tools that are needed in order to accurately predict which candidates will be most successful. So what type of information does a good foundation have? While the specifics differ based on the job and the situation at hand, all foundations contain two basic types of information. I refer to these as “above ground” information and “below ground” information. Above Ground information Above ground information includes aspects of the job that are relatively straightforward and easy to identify and evaluate. Common examples include:

Minimum qualifications (e.g., years of experience, level of education, willingness to relocate)

Basic stuff one has to know in order to do a job (e.g., what computer programs are required, or what basic background knowledge is required)
Objective job requirements (measurable criteria such as sales quotas)

Above ground information is pretty straightforward. It is usually identified simply by reviewing job descriptions or speaking with hiring managers. This type of information is most useful for providing a foundation for top of the funnel, non-scientific screening tools such as application blanks and resume evaluations. Below Ground Information Below ground information involves aspects of job performance that may not be readily apparent from initial non-scientific screening activities. These are the things about an applicant that you can’t measure unless you are using the correct tools. Below ground information includes:

Personality traits (e.g., adaptability, flexibility, conscientiousness)
Abilities (e.g., verbal ability, mathematical ability, logical reasoning)
Skills (e.g., interpersonal skills, oral communication, situational judgment)
Fit with job environment (e.g., work requires a high degree of autonomy, work occurs in a decentralized environment)

Just because you can’t see this type of information, that doesn’t mean you can ignore it. Just like with icebergs, what that you can’t see is often the biggest (and most dangerous) part — failing to plan for what you can’t see can prove disastrous. While adding below ground information to a foundation takes a bit of effort, the importance of building a comprehensive foundation cannot be understated. The quality of the information provided by the foundation-building process is critical because, no matter how good a screening tool may be, it is impossible to make good hiring decisions without a complete understanding of what is required for job performance. How Do They Do It? In theory, creating a foundation is no more complicated than 1) identifying above and below ground information that defines job performance and 2) verifying the accuracy of this information. But in reality, it is a bit more complicated. There are some specific legal guidelines that must be followed when building foundations. Unfortunately, following these guidelines to the letter requires a great deal of time and effort, a situation that is not compatible with the hiring goals of most organizations. This has led vendors of online screening to “interpret” the rules for foundation building when creating their products. Each vendor’s interpretation represents a balancing act between providing efficiency and satisfying legal requirements. Add to this the fact that the legal defensibility and effectiveness of each foundation-building process depends on the parameters of the situation, and you can see how building foundations can become a complicated issue. In order to try and simplify things, I have broken down online foundation building into five basic processes. Most systems use some variation of one of these five processes, and many use more than one.

Study of existing job materials and input from stakeholders. This method is best suited to providing above ground information. Known information about the job is examined by the people doing the hiring, and a set of screening criteria are developed. Many online screening systems offer tools to help system users to build foundations based on this type of information. Some systems even include an application or utility that extracts relevant information from job descriptions or a proprietary database. These systems are adequate for coarse, top-of-the-funnel screening; however, they are insufficient for providing below ground information.
Generalization based on existing data. These types of systems utilize some form of previously collected data to build screening foundations. This can take the form of existing organizational competency models, databases of job requirements, such as the O*net, or the collected results of organizational validation studies. These types of foundation-building tools provide a predefined profile that is supposed to define performance for a job or family of jobs. With these systems, all that is needed is to verify that the job in question has the same characteristics as the preexisting profile and you’re off to the races.
Confirmatory surveys. These tools involve the use of a survey that contains information about what is required for a specific job. Members of the client organization are asked to complete the survey, and the results are examined in order to confirm that a specific screening tool is adequate for a given situation. The contents of the survey are usually tightly linked to the content of a set of predefined screening tools.
Employee benchmarking. Some systems create foundations for screening by asking high-performing incumbents to complete the screening measure in question and aggregating their scores to develop a benchmark. It is assumed that the applicants most closely matching this profile will make the best employees. Applicants are then screened in or out based on how well they match this profile.
Job analysis study. There are many types of job analysis studies, but the basic process involves some form of organizational research study. These studies usually involve the review of existing job materials, interviews with job incumbents and their supervisors, and several surveys. This process results in a firmly defined and comprehensive blueprint that tells you the relative contribution to job performance made by each major component identified in the foundation. The intensity of the job analysis process varies quite a bit, but most true job analyses follow a well-defined set of rules in order to arrive at a foundation. Most of the time following these rules involves purchasing consulting services from a screening vendor.

Some Basic Requirements It is probably no surprise that there is no one absolute best method for building foundations. Choosing the best option depends upon situational parameters and organizational needs. In reality, many screening systems will use a combination of methods and many often provide the option for conducting a more in-depth job analysis to support more cursory methods. In my opinion, when it comes to foundation- building tools the issue is one of the ends justifying the means. That is, as long as the process yields realistic information and provides some mechanism to verify its accuracy, then it is probably acceptable. This does not mean that I am condoning cutting corners. I firmly believe that all foundation-building processes (especially those used to obtain below ground information) must meet some basic requirements. These include:

A solid confirmatory process. The most important requirement for all foundation-building processes is that they provide a way for organizational members to verify the accuracy of any screening blueprint that is created. This verification should come in the form of data that provides documentation of the accuracy of the foundation. This process should also provide a mechanism for aggregating the information gathered so that the consistency of the data can be examined.
Multiple sources of input. It is best to get verification from people holding different roles within the organization. In most cases, it is not a good practice to have only one hiring manager or recruiter verify a foundation. It is important to seek verification from incumbents, supervisors, and higher-level personnel.
An adequate sample. All verification surveys should be completed by a reasonable number of people. Having four or five people participate in verification is less than ideal. As a rule, it is important to get a sample of about 10% of the incumbent population to provide verification information. It is also a good idea to make sure the sample you use is representative of the overall population in terms of race, sex, age, and job performance. I realize that it is not always possible to obtain an adequate verification sample, but in such cases it is important that the screening system provide additional methods to ensure the accuracy of foundations.
Data. The use of any off-the-shelf blueprint must be supported by some form of data. Productized screening solutions that are created for specific types of jobs (e.g., sales, entry-level, retail) should provide data that demonstrates that a foundation has worked in other similar situations. In addition, all screening systems using this type of foundation-building process should provide a way to verify that the position in question actually shares the same critical aspects of job performance with the blueprint.

The Future of Foundation Building At the present time many of the foundation-building processes used in online screening systems require some form of verification process. Such verification is critical but presents problems because it can be time consuming and often requires the use of consulting services. The future will see a move away from the consulting model of foundation-building and towards a more automated model. This movement is an essential step in the development of the new paradigms for of online screening. As we move forward, the main issue that must be addressed is automating the verification process without losing the ability to ensure its accuracy. Reducing human involvement in the verification process will require screening systems to have built-in intelligence that frees users from having to make decisions that they are not qualified to make while still allowing them to quickly create highly accurate screening foundations. The development of these systems will require the collection of data that allows the construction of shared databases containing building blocks that can be easily configured to define job performance for a wide variety of jobs. This will clearly take a lot of work… so let’s get moving!