Simulations: A Look Into the Future of Scientific Screening

The proliferation and evolution of the Internet has created momentum that is slowly moving the entire hiring process online. This trend has led to the development of some really interesting ideas, many of which have been turned into products and systems that are helping both job seekers and employers learn more about one another. Despite the fact that current online hiring technology is still pretty crude, the impact of these systems has done nothing less than to completely change the way that both job seekers and employers think about the hiring process. Keep in mind that a five to ten year period is all it has taken for this to occur. Given this perspective, I don’t think we can even begin to imagine what the online hiring landscape will look like 10 years from now. But even though I can’t make predictions about the impact of technology that hasn’t even been invented yet, I can make one prediction with great confidence. This prediction is that in the future, simulation-type assessments will be a common tool used by organizations to measure, evaluate, and predict a candidate’s ability to perform a wide variety of jobs. Why am I so confident in this prediction? At the present time, the bulk of online assessment involves the delivery of the same screening tools that have been used for years using the Internet. While using the Internet certainly has many obvious advantages over past methods, this model is really only a new way of delivering the same old package, and falls very short of using the present level of Internet technology to its full capacity. The evolutionary forces impacting the development of online assessments will eventually force us to move past this model and to develop entirely new paradigms for online assessment. The predictive power of simulations and their compatibility with the technology that makes the Internet work almost mandates that these new paradigms involve interactive simulations. The remainder of this article is dedicated to providing the information needed to support my prediction. What Are Simulations? Simulations include a variety of different types of assessments. The common element in all of these is the re-creation of situations or tasks that are critical to the performance of a specific job for the purpose of predicting an applicant’s ability to perform the job in question. Simulations are used for two basic purposes: to assess an applicant’s aptitude for learning how to perform tasks required on a given job, or to assess a candidate’s present ability to perform job-related tasks. The simulations used for each of these purposes differ slightly, but function in the same basic way in that they are based on the same set of scientific principles used to construct all assessments used to predict job performance (please see my previous articles for a more detailed discussion of this topic). The most common types of simulations include:

Work samples. The work sample is the most classic example of a simulation used for assessment. Work samples are commonly used to assess physical aspects of job performance such as strength and dexterity. Work samples gained popularity in industrial settings where applicants were often asked to demonstrate mechanical ability or dexterity by completing assembly tasks that closely represented those required on the job.

In-baskets. These exercises place candidates in a situation where they are asked to assume a role related to the position they are applying for. In-baskets inundate candidates with information that must commonly be dealt with on the job and require that they quickly make decisions based on this information. The manner in which the candidate processes this information and the quality of the decisions made are then evaluated by experts using an objective scoring system.
Simulators. Simulators are machines or computer programs that re-create the environment found at a job. Often these environments are simplified so that candidates can use them without in-depth knowledge of a job. Candidates are asked to complete tasks using the simulator and their responses are measured and scored. A good example of a simulator is the computer-user environment used to assess candidates applying for call center jobs. This environment requires candidates to deal with incoming calls and perform tasks using an interface that is a mock up of a simulated call center.
Skills testing. Not all skills tests are simulations. The type of skills testing I am referring to is that which asks candidates to use their knowledge of a particular computer program or language in order to solve problems similar to those they might face on the job. For instance, a candidate for a Java programming job might be given a simulation in which they are asked to create or debug an application that performs a certain task. These simulations are most commonly used to help measure how well a candidate is able to apply a certain body of knowledge to solving problems faced on the job.

Why Are Simulations So Powerful? Simulations have been shown to be highly effective and valid predictors of job performance. There are a variety of reasons why simulations are one of the most powerful tools available for selecting employees. These include:

Job relatedness. The major reason for the effectiveness of simulations lies in their job relatedness. Since the start of time, the litmus test of the value of any assessment used for employee selection has been the job relatedness of that assessment. The closer the information gained via measurement approaches the skills required for successful job performance, the more confidence one can have in making predictions based on that assessment. This is known as the idea of “point to point correspondence” between a predictor of performance and the job in question. Simulations offer a very close match between assessment tasks and tasks required on the job. This relationship also means that simulations provide excellent legal defensibility when used as assessment tools (provided that they are properly constructed and validated).
Face validity. The concept of face validity means that the contents of an assessment tool appear to be related to performance at a given job. Simulations have very high face validity because they include tasks that closely resemble those faced on the job. This is important, because candidates are often upset by assessments that do not appear to be related in any way to the job they are applying for. Negative impressions due to a lack of face validity can cause a variety of problems including: damaging an employer’s brand image, causing candidates to withdraw from the selection process, and increasing the probability that an applicant will seek legal action due to dissatisfaction with the results of an assessment.
Simulations are fun and engaging. Not only do simulations resemble the tasks required for a job, they can actually be fun for applicants. This fun factor can have a huge impact on an employer’s ability to use an assessment to help build an employment brand. The ability to offer an assessment that simultaneously provides value for the job seeker and the employer is a winning situation for both parties.
Realistic job preview (RJP). The similarity of simulations to tasks found on the job means that they can actually provide applicants with a realistic view of what it is like to hold the job they are applying for. This is important because the information provided by the RJP allows applicants to better understand what will be required of them on the job. This allows persons who do not feel qualified or who do not enjoy performing job-related tasks to remove themselves from the selection process early on, a situation that can help to reduce turnover.

The Present State of Evolution for Simulations At the present time there are very few actual simulations that are delivered electronically. I am not aware of any true web-based work sample tests, and although the use of in-baskets is relatively widespread, there are currently no true electronic in-baskets available. Simulators and skill assessments are the most common types of simulations currently available using electronic delivery methods. These simulations focus mostly on testing candidate’s ability to write code in a specific language or use a particular program to solve work-related problems. There are also a variety of very effective, truly interactive call center simulations and non-interactive, low fidelity simulations currently available. While these simulation options are effective and represent the cutting edge of current technology, they still fall way short of what simulations are capable of and what they will look like in the future. This does not mean that these simulations are ineffective. I think that anyone who has used them for the purposes of employee selection would be happy to admit that these tools have made a big difference in their ability to select qualified applicants. So What Will the Future Look Like? The future of online simulations to be used for assessment is very exciting. The Internet is the perfect vehicle to deliver highly realistic and entertaining simulations that will really advance the way people apply for jobs. The ability to construct artificial environments and use streaming video and audio linked to branching technology and sophisticated databases are the ingredients needed to construct and deliver a whole new class of assessments meant to measure work-related skills and aptitudes while also entertaining applicants. It is entirely possible that these assessments can look and feel just like a video game while delivering high quality, job-related assessment. The most exciting possibility is that these assessments provide us with the tools we need to move past the traditional definition of simulations. The knowledge we accumulate about other softer skills such as personality-related skills can be woven into these assessments so that only one simulation is needed to provide integrated measurement of both hard and soft skills. What can we be doing now do to help promote the development of simulations? Admittedly we are not close to my vision of the future yet, but as I write all the pieces of the puzzle needed to make truly innovative solutions currently exist. Programmers can make amazing interfaces, I/O psychologists know how to use science to develop job-related assessments, and database specialists can create sophisticated data collection, storage, and branching algorithms. The major issue that has limited the advancement of simulations is the large amount of money needed to fund their R&D. With the development of even a simple web-based call center simulation costing close to a million dollars, it is no surprise that the development of new simulation models is slow now. It will take a more universal understanding of the value of Internet-based screening to remove this obstacle. But here are some things we can be doing now to help foster the development of the future of electronic simulations.

Document the value of online assessments. Use currently available web-based screening tools and document the value they add. The more value seen in these tools, the easier it will be to push for the additional funding needed to develop new technology.
Think outside the assessment box. The value of a slick, fun tool to help match candidates with jobs is a win/win situation for both parties. It can help a company make major strides in terms of building an employment brand and developing a winning corporate employment portal. This is a sales pitch that can be taken to branding and marketing departments. These folks usually have much bigger budgets than HR does, so partnering with them can help distribute the cost of a cool simulation project.
Don’t be afraid to push for innovation. There is a tremendous premium on being first to market with new and innovative interactive assessments. Focusing on these payoffs can help provide momentum for new simulation projects.
Create partnerships. Develop partnerships that will help distribute the cost of producing simulations. Partnering with firms that have the technology needed to help develop new and interesting simulations can help ensure that diverse skill-sets are used to help develop new products while also helping to defray costs.

It is not a matter of whether simulations will evolve to become a prominent type of online assessment technology, but a matter of when. The more we understand about the value added by the simulations we are currently using and about how we can make these better, the faster we will be able to reap the benefits of an entirely new class of assessment tools.