Receive daily articles & headlines each day in your inbox with your free ERE Daily Subscription.

Not logged in. [log in or register]

The New College-grad Test Is Better Than GPA, But Proceed With Caution

by
Dr. Wendell Williams
Aug 29, 2013, 6:45 am ET

cae-logoAs long as I have been in business, hiring managers have been trying to pin a magic number on job candidates in the hope it will indicate future job performance. Sometimes that number is a GPA, combined test score, or even past earnings. Now we have the Collegiate Learning Assessment (CLA+) … a test given to new college graduates and scored like the SAT. But guess what? Magic numbers are just that … magic!

Magic is the art of illusion. That is, a magician creates illusions using sleight of hand that deceives the audience. Hiring decisions based on a number are sleight of hand because they lead people to think everything about a person can be reduced to a few digits. But anyone with enough years on the job and the professional savvy to systematically compare job performance to pre-employment test scores (i.e., studies … not stories) knows this is only part of the performance story.

Hiring scientists have known for years that GPAs only predict future GPAs; so, according to the WSJ, a professional test vendor has developed a new test for colleges to use that measures new-graduates’ critical thinking ability, problem solving skills, scientific and qualitative reasoning, writing, and the ability to critique and make arguments. Well, as a professional test developer, that sounds pretty much like a cognitive ability assessment battery to me (don’t get spooked, “assessment” is just a fancy word for testing or measuring a candidate’s skills).

The Benefits of Cognitive Ability Tests/Assessments

When a job — any job — requires skills like solving problems and making good decisions, arguing that people who are smart enough for the job will outperform those who are not is a no-brainer, especially when a test is as comprehensive as the CLA+ claims. After reading their technical manual, the CLA+ publishers seem to have done all the necessary homework required of a reputable test developer. Any user wanting to buy a hiring test (i.e., assessment) would do well to read their development process (http://cae.org/). So what could possibly go wrong with a test like the CLA+?

Validity and Reliability Claims

For starters, no test vendor can assure a user their test will work as promised unless and until scores can be shown to predict job performance for their company in their job (note: I am not talking about group-match nonsense). As you can imagine, many test vendors have major problems with this concept and expect you to take their word as absolute truth; after all, they have tests to sell and don’t live with the bad-hire expense.

If you don’t validate the test for your job, you will have no idea if what works for your accountants works for your salespeople, or if what works for your clerical staff works for your production workers, and so forth. True, the CLA+ meets professional test standards, but the real question is whether it predicts job performance in your organization for your job. (Note: if validation work has already been done for a parallel job, and if the new user can prove the two jobs are highly similar, an organization can transport the validity data. Otherwise, you have to do it yourself). There is no free lunch when it comes to using hiring tests.

Mandatory Homework Assignment

There is a long track record of adverse impact associated with cognitive ability tests. Minority groups (not individuals, mind you) tend to have different averages (again, this does not apply to individuals who can score anywhere). Group averages immediately attract political attention because low-scoring demographic groups also happen to be voters.

HR steps constantly right into the adverse-impact stuff because it seldom does the necessary homework to identify and clarify exactly what scores work for a specific job, what are too high, and what are too low. In addition to making payments on several lawyers’ Mercedes, an HR department that does not do its homework is likely to either hire a whole bunch of smart people who quickly get bored or hire a wide collection of two-legged productivity anchors. Either way, the organization loses and there is only one department to blame.

Multi-Dimensional

How many brilliant people do you know who are also good organizers? Have effective people skills? Are highly motivated? If you agree that job performance takes more than just being smart, you get the picture. Many jobs require a broad array of skills. Some require above-average interpersonal skills such as coaching for managers, relationship building, and fact-finding for salespeople, as well as resolving complaints for customer service. Other jobs require special organizational skills that go beyond time management such as attention to detail, project planning, and prioritizing. And all jobs require motivational components so employees will actually want to use their skills and abilities.

Assuming a single score will predict overall performance overlooks how each human dimension affects job performance and assumes all skill dimensions are equal. No, personality tests do not accurately predict job skills. Nor is there any assurance that personality scores are even accurate. If anyone wishes to challenge this statement, please send me your data showing which personality score has a strong correlation with a specific verifiable skill and I will write an article about it.

Career Planning

Dull people are like parasites who contribute little and take much; but, what do you call a whole bunch of smart, highly skilled people who all want to be CEO in a few years? Do you have a plan to keep them happy and well fed (other than leaving the company for a competitor)? You see, when you start testing for specific skills, you will get people who have them. And a company filled with very bright people with nowhere to go usually has high turnover among both good employees and managers.

What’s Next?

Unless you hire everyone who applies, you are in the assessment business. Better than an interview, the CLA+ is clearly a step ahead of GPA scores because it measures a well-researched and proven predictor of job performance. But HR needs to look further before considering the CLA+ to be a complete hiring solution.

Knowing that hiring managers want to categorize applicants into little boxes with single digits means HR has to ensure every test (e.g., assessment) score is properly used, do the necessary homework to set scores properly; insist it be only one part of a larger hiring and promotion system; monitor adverse impact at the group level; and, do the necessary job analyses work to justify using a cognitive ability test in the first place.

Remember, HR is in the assessment business. The raw materials that get through their screen have direct effects on quality, turnover, productivity, legal challenge, and profitability. And that’s a fact.       

This article is provided for informational purposes only and is not intended to offer specific legal advice. You should consult your legal counsel regarding any threatened or pending litigation.

  1. charles handler

    Wendell,
    Nice take on this.
    I have not yet heard of this test, however I do have pretty extensive experience with the concept. There are a # of firms now looking to do this kind of thing (bloomberg for example). I think the value lies in the high level ability to have a reliable and standardized data point that can be used as part of a matching equation to support screening at volume. While the vagaries of this type of test are many, one also has to look at the context and the problem it will solve. That is supporting a way to sift through huge volumes of data using some meaningful and standard screening tools. If used this way, I think it has value. It is not something that one would want to use as final decision making tool, nor do I think its creators would advocate this.

    I plan to learn more about this test and write about it here on ERE soon.

    Thanks for bringing it to light and sharing what I believe to be an excellent viewpoint on it.

    Charles

  2. Tom Fosgard

    I have mixed feelings on the concept. When screening new college graduates, it ends up being a soft skills interview rather than a knowledge-based screening that I’d expect this test to evaluate. College grad interviews are gauging maturity, communication skills, work ethic, aptitude and team orientation. I do believe that grads that do well on the test probably are exhibiting maturity, work ethic and aptitude in some way, which is good and could help narrow the large pile of resumes I receive for these positions but I’m not entirely sold on the idea.

Post a comment

Please log in to post a comment.

Note: You need to sign up for an account on our new commenting system if you haven't already done so — even if you have an existing ERE account. Find out why »

Login Information