Assessment! What a concept! Imagine a world where job applicants are screened for their job skills?before being hired! Wow!
Assessment = Judgment = Test = Interview = Application = Resume
It’s so simple, it’s complicated.
Folks, anyone who screens resumes or applications, conducts interviews, or reviews past work history to predict future job performance is already using assessments. Ads, postings, websites, and referrals may bring applicants to your doorstep, but assessments separate employees from wannabes.
So wake up and smell the coffee. Assessments are not weird, foreign, or unusual. They are used every time someone places an ad in a trade newspaper, posts on a job board, conducts an interview, uses a “smart” Internet application, “sells a pencil,” or gives a test.
The sooner we realize our profession is up to its armpits in the assessment swamp, the sooner it can change our image from a “learn-as-you-earn” job to one where highly skilled professionals command respect. Can anyone escape the assessment swamp? Sure: advertise everywhere and hire anyone who applies.
When we post ads to a specific newspaper or website, or ask for personal referrals, we are using a source-related assessment. In other words, we expect these sources to minimize unqualified job seekers.
For example, posting on an Internet board excludes people who are not computer-savvy enough to navigate the Internet and post to the site. Posting in the WSJ excludes people who are not business-savvy. Posting to a site catering to a specific ethnic group excludes everyone who is not a member of the group.
You get the idea. Sourcing choices are a subtle form of assessment because they act as not-so-subtle pre-qualification screens.
A recruiter or hiring manager might think, “Hmmm. Interviews are ways to get to know someone. If I get to know you, you should be able to do the job.” Well, if our objective was to know someone at a surface level, then interviews would be appropriate.
However, anyone with a few years of career experience knows that interview skills and job skills are two entirely different things. As we all know, even the most likeable people can turn into incompetent employees.
The problems with using interviews as assessments are well-known: the interviewer often has an unclear idea of the skills required; question techniques might be leading or unclear; the applicant may not be able to recall a good response; applicants fib; personal appearance may affect the decision; and so forth.
In a desperate attempt to add credibility to the interview, people advocate wrong-headed questions like the “gotcha!” (e.g., when candidates are invited to say something negative about themselves); the “one-question-wonder” (e.g., when interviewers have a secret internal job-standard against which everyone is measured); or the “pseudo-shrink” (e.g., where the interviewer thinks he or she is a practicing psychologist and asks deep questions like, “If you were a tree, what color would you smell like?”).
Yes, interviews are assessments. They have questions, answer guides, and pass/fail standards. Once you screen out bottom-feeders, they tend to be about the same as chance at predicting performance.
Automated assessments usually include some form of computer interface that presents applicants with a situation/question/simulation/application blank, asks for a response, and scores the answer. The assessment could be located in a kiosk at the front of a store, on a computer in an office, or even presented through the Internet. Its purpose is the same as the interview: screen-out unqualified applicants.
Does it work? Maybe. It is probably better to think about how well it works. Garden-variety interviews have almost no predictive power. Structured interviews are better because they are more highly focused. Tests and simulations are among the best predictors because they are exceptionally hard to fake.
But that’s not all you need to know about assessments. You need to master the dreaded validity beast.
Before I started this profession, I was certain that motivation test scores were closely associated with intelligence. That is, I was convinced that high/low scores on motivation tests were valid (there’s that word again) predictors of intelligence. But after a few studies, I saw that 99% of the time, motivation scores turned out to be wrong. I should have known better.
Do you really know whether your test/assessment/interview predicts job performance? I don’t mean war stories; I mean solid investigations to determine whether the applicant is qualified or unqualified.
What can you do to ensure that test scores are valid? First, understand that “performance” can mean a lot of things.
Some people (professional and otherwise) get confused about performance. Is performance the number entered on performance appraisals (which everyone lies about anyway)? Is it the score at the end of a project when everyone gathers to take credit for success or assign blame for failure?
Or is it something else?
It helps to think of job performance as a fruit salad: little bits of pears, apples, peaches, pineapple, and grapes residing in a bowl. Each fruit represents a different set of skills.
Pears equal cognitive ability or the employee’s ability to analyze situations and make good decisions; apples represent the ability to learn and apply new information; peaches symbolize planning and implementation skills; and so forth. Each piece of fruit represents a job-related skill that, when used properly, contributes to overall job performance.
Most folks tend to put individual skills into a blender, then press the annihilate button. Within seconds, individual skills disappear into a performance puree. In the process, they lost the ability to examine individual pieces and have no idea whether the pears were exceptional or the peaches unsatisfactory.
A valid test requires comparing apples to apples: that is, on-the-job learning to learning-ability tests; on-the-job persuasiveness to persuasiveness simulations; on-the-job planning to planning tests; and so forth. Comparing fruit puree to 35 dimensions of personality, an industry job “norm,” or a specific test may produce data that looks and sounds credible, but it can produce nonsense numbers.
Performance is not a single dimension. It is the score at the end of the game, the baby at the end of the pregnancy, or the black eye at the end of the long argument.
Someone once argued that his assessment system was so simple, it did not require a Ph.D. to use. I suppose this person thinks this is a good thing. Hopefully, after reading this article, you will realize that “simplicity” and “highly effective” don’t play well together.
You may not need a Ph.D. to be a competent assessment professional, but if you expect your assessment/test/application form/website/interview to accurately predict job “performance,” you better think like one.