Don’t you just love it when people think they are test experts? You know, with all the public posturing about being professional, being a strategic player, and adding value to the organization. Then reality hits ó and it hits hard! Yep. I’m talking about using a totally inappropriate test to hire people, again! This particular case started in a chain of retail stores that rent furniture, electronics, computers, and appliances. Someone there probably looked around, decided their turnover was too high or productivity too low, and said, “Gee, why don’t we use the MMPI to test candidates?” Maybe they attended a workshop. Or maybe a senior executive talked to a neighbor who talked with a golf partner. “Swell shot, Bob. Have you ever thought about using a mental illness test to hire employees?” When executives make test decisions, head-bobbing subordinates often risk severe frontal-lobe damage from nodding so hard in agreement. Too bad for the head-injured, when the dreck impacts the wind impeller (and it will), the same executives who suggested a litigation-prone test in the first place will dust off their own finely-honed survival skills to publicly skewer the head-bobbers who “should have warned them.” What’s frequently missing from the way tests are usually chosen? For starters:
- A job analysis
- A technical test review
- A validation study
- Expert advice from an experienced industrial psychologist.
Back at the Ranch The test that’s the subject of today’s article is the MMPI, which stands for Minnesota Multiphasic Personality Inventory, a 300- to 500-item multiple-choice diagnostic test. According to its publisher, the MMPI-2:
…is the most widely used and widely researched test of adult psychopathology [i.e., nutso-behavior]…used by clinicians [i.e., people who treat mental health problems] to assist with the diagnosis of mental disorders [i.e., craziness] and the selection of appropriate treatment methods [i.e., mood altering drugs or institutionalization]. It helps meet the assessment needs of mental health professionals [i.e., clinicians] in an ever-changing environment.
That’s the publisher’s expert description. Now, some additional background information about this particular test:
- The MMPI is more that 65 years old.
- It was developed to help clarify categories of diagnosed mental illnesses.
- Items and scores were based on responses of clinically diagnosed mental patients treated at The University of Minnesota Hospitals.
- Items and scores were cross-validated based on responses of patients’ relatives and visitors (yeah, as if Uncle Nutso’s family members represented a “normal” population of job applicants).
- Items included questions about sexuality, bodily fixations, hallucinations, delusions, gender roles, and all the other interesting things that characterize mental illness.
Considering the publisher’s information and the history surrounding the MMPI, even the least test-wise reader might note that “predicting job performance” or “hiring retail employees” is NOT among the MMPI’s recommended uses. Here Comes the Judge! We don’t know all the details of the case, but it seems the test wizards at this chain of furniture, appliances, electronics and computer-rental stores routinely used the MMPI to select people for management positions. Now, unless the organization intends to rent a line of Martha Stewart Designer Weapons of Mass Destruction, any reasonable person would see the MMPI is the wrong test. To emphasize the point, I ask the reader to imagine a line-up of the principals in this case:
- In Position One we have a representative of the executive suite who thinks the company is using the right hiring tests.
- Position Two represents the internal test wizard(s) responsible for choosing and administering the hiring tests.
- Position Three represents the star-crossed candidates who take the test.
- Position Four holds a representative of the shareholders and owners of the company.
Question: Which representative is most likely to carry an official-looking clipboard, wear a white lab coat, and sport big red floppy shoes? Any legitimate test has three parts: 1) something to legitimately measure, 2) a series of legitimate questions and, 3) a standardized score sheet. How can the reader tell if his or her test should be used for hiring? First Clue: Ask the Vendor! Fortunately for test vendors, the law does not generally hold them responsible for test misuse by people who wear big red floppy shoes. And any test user who thinks their test publisher will “rush” to their defense is in for a rude surprise. As the 1978 Uniform Guidelines explicitly states:
Article Continues Below
Under no circumstances will the general reputation of a test or other selection procedures, its author or its publisher, or casual reports of its validity be accepted in lieu of evidence of validity. Specifically ruled out are: assumptions of validity based on a procedure’s name or descriptive labels; all forms of promotional literature; data bearing on the frequency of a procedure’s usage; testimonial statements and credentials of sellers, users, or consultants; and other non empirical or anecdotal accounts of selection practices or selection outcomes.
The first step toward losing the big red floppy shoes? Before using a test, ask the vendor a simple question: “Was your XYZ test (or interview guide) developed for hiring purposes, and does it explicitly predict future job performance?” Possible answers:
- “No. But, it can help you make hiring decisions!” Translation: Fertilizer alert! How can a test that was not developed to predict future job performance be used to predict future job performance?
- “No. But, it can help supplement interview data!” Translation: See Answer #1.
- “No. But it can help you make a hiring decision.” Translation: See Answer #1.
- “Yes!” Translation: This is the answer you want to hear. Proceed to the next clue.
Second Clue: Examine the Items Don’t be tempted into second-guessing an experienced professional test designer unless you are also qualified to teach a graduate-level course in psychometrics. Here are some things to look for:
- Know that hiring tests always contain a “secret formula.” Questions on hiring tests should be are job-related, but that does not mean every item is going to be politically sensitive. Virtually all legitimate test designers include in-your-face questions like, “Did you steal your last company blind?” and “Do you take home office supplies?” However, since professional designers know that a certain group of applicants will actually agree with questions like this, they build their tests so that scores are normalized and multiple items are used to stabilize responses.
- Business-related items are a “good” thing and personal questions are a “bad” thing. This is called “face validity.” Oh, yes, you might also want to be acutely aware that some countries and a few states might take issue with personal questions. Always be sure to consult a good local labor law attorney for advice (one who understands psychometrics).
- Never forget, while training tests are “fun” and designed to “please,” hiring tests are often in your face and intended to deliver bad news about the test-taker. So don’t be tempted to take a hiring test yourself unless you have stamina for digesting unpleasant information. Good hiring tests are backed by norms and standards, and none of us are “normal” in all categories. “Try it, you’ll like it” only works when a test taker aces the test. Otherwise, be ready for negative feedback.
Final Clue: Make Sure the Test Works for Your Job There are good reasons for a formal validation study. Performance is usually messy, and relationships are based on probabilities and major trends, not perfect linkages. For example, we know the majority of high-performing people are smart, but we also know that not all smart people are high performers. Is there a strong causal link? Absolutely. Do “smarts” always predict job performance? No. However, in spite of all the problems, we owe it to both the candidate and the organization to make sure every hiring test score is “tweaked and tuned” for each specific application. By the way, if you see anyone around the office sporting a lab coat, clipboard and wearing big red floppy shoes, show him or her the door and find a professional to help get back on track.