Last week the Wall Street Journal ran a very interesting article: “Test for Dwindling Retail Jobs Spawns a Culture of Cheating.”
This well-written and researched piece is somewhat groundbreaking in that it is the first article in a mainstream media outlet to provide evidence of cheating on employment tests. Those of us in the testing industry have always been concerned with the security of our tests and have taken a variety of precautions to defend against it, but this is the first time I have ever read actual evidence that documents the existing of cheating.
As a testing expert and someone who has a high degree of familiarity with Unicru/Kronos (the company whose tests are the subject of the piece), I figured it would make sense for me to weigh in on this important article. Here are some thoughts about the article and the issues it raises:
- Overall, the article should definitely serve as a wake up call to the testing industry that times are changing; but, I do not think that the problems it identifies should spell doom and gloom for employment testing. Security is an important issue for any industry that conducts business online (banking, shopping, etc). However, dealing with security issues has not served as a barrier to progress, but rather an evolutionary force encouraging adaptation and prevention.
- The case of Unicru/Kronos is somewhat unique in the testing industry because it tests such a huge volume of applicants each year. It is a safe bet to say that no other testing company does the volume that Unicru/Kronos does. Therefore I would be very surprised if the same systematic and premeditated cheating on personality-type tests was occurring with the tests of any other vendors. Differences between the way the Unicru/Kronos tests and those of other companies would make cross-test cheating a complete disaster for anyone who tried.
- Because the Unicru/Kronos test is used to provide a way to provide efficiency to high-volume hiring, its results are often used to make “go — no go” decisions. While it is not always advised to use a test as the only piece of information to eliminate an applicant from consideration, high-volume hiring often necessitates it. How else can one have any chance of evaluating 1,000 applications for 10 or 20 openings? Considering the current state of unemployment and the sheer number of folks involved, high-volume hiring starts to show some similarities to standardized testing. That is, lots of applicants and high stakes. While cheating on high stakes tests is not too common in the U.S., it is a significant problem in other countries such as China and India. I have had a good deal of experience implementing standardized testing in these countries and I can say with authority that we do have a variety of techniques that can be employed as preventative measures. As with any type of security, multiple layers provide the best defense. We as testing professionals have the ability to use item banks, alternate versions of items, parallel forms, and computer adaptive testing to minimize the chance that a reconstructed answer key will actually match a test.
- Consider the type of test in question when evaluating the potential for cheating and the impact it may have. The WSJ article claims the Unicru/Kronos test is a personality test. This is partially true, as it does have some strong ties to traditional personality tests. However, the most common personality tests do not actually have right or wrong answers for the individual questions. Rather, the scoring for the questions usually varies quite a bit relative to the specific job in which the test is being used (if the test has been properly configured). While the science behind Unicru/Kronos’ tests is beyond reproach and is actually very innovative, their tests are still a good bit different from those of the rest of the herd. Unicru/Kronos’ unique formula works for them and their tests are highly predictive when compared to many others out there. Why does this matter? I won’t go into the mechanics of it, but suffice it to say that it would be much harder to identify right or wrong answers on a more traditional personality test. Most traditional personality tests also have faking scales built in to identify those applicants who are answering in a socially desirable pattern. Figuring out the scoring algorithms that account for this makes cheating in any meaningful way even more difficult.
- Traditional personality tests by themselves really aren’t the best predictors of job performance. There is a ton of research to support the fact that cognitive ability tests are much better across the board at predicting job performance. These types of tests are much more susceptible to cheating, and test vendors are very aware of this fact. In many cases, these types of tests are administered only in proctored situations. If I were to really worry about cheating, it is cheating on cognitive tests that would be my major concern.
- Another interesting wrinkle in this whole thing is the fact that job applicants are actually resorting to cheating the companies that they wish to potentially work for. To me this is a clear indicator of the need for companies to take a good long look at not only the messaging around the test itself, but also at the manner in which the applicant is treated throughout the entire hiring process. Of course it is unrealistic to think that we will ever be able to keep all applicants from cheating, but I do believe that the more applicant-friendly the hiring process, the less likely applicants will be to cheat. Time and again I have seen the testing portion of the application process handled in a cold and unfriendly manner, with very little information about the reason for the test being provided to the candidate. This type of thing makes it much easier for a candidate to feel fine about breaking the rules. There should be messaging to candidates about the fact that testing is beneficial to them because finding a good fit between applicant and job is beneficial to all parties involved. No one likes to work in an environment where they do not fit in or where they do not have the competencies required to succeed.
- The bottom line here is that tests such as those offered by Unicru/Kronos do offer tremendous value. We I/O psychologists have collected more than enough evidence to demonstrate that testing, when done right, offers a huge advantage over more informal and unstructured hiring practices such as resume reviews and unstructured interviews. In the case of high volume hiring, the ability to provide an automated tool for screening out applicants is tremendously useful. Are these methods perfect? Absolutely not, but then again very little in this world is. Error and noise are accepted parts of the testing process. The goal of test providers and hiring professionals is to mitigate these to the best of their ability. In high-volume testing, cheaters make up a very small percentage of the total population of job applicants, making cheating just another source of noise that we must work on minimizing.
The issue of cheating is something that warrants a good deal of attention but one that should not stop our progress in the quest for better and more accurate hiring procedure. The issues discussed in this article serve to strengthen my stance that we need to be working on a new generation of hiring tools that are simulation-based. The more the testing part of the hiring process relies on scenario-based activities that mimic the work environment and the tasks that must be completed within it, the more realistic and engaging the application process will be for applicants and the harder it will be for someone to cheat. Of course we are a long way away from this goal at the present time, but we will get there eventually. Until that time, those using testing should be sure to make test security a critical part of the design and implementation of the testing process, while also bending over backwards to make sure the candidate is treated just as any valued customer of the organization would be.