Receive daily articles & headlines each day in your inbox with your free ERE Daily Subscription.

Not logged in. [log in or register]

The Problem With Personality Tests

by
Dr. Wendell Williams
Jul 12, 2013, 6:03 am ET

It seems counterintuitive. Some people claim that personality has everything to do with job performance. Others know that every piece of serious research shows personality scores have almost nothing to do with job skills. I’ll repeat that: personality scores seldom, if ever, equal skills.

Soooo, if personality has nothing to do with skills, why do so many organizations used it to predict job performance? The answer depends on how and where you look.

Predictive Accuracy

Let’s start at the beginning. Researchers agree that adding a decent job-related developed personality test will improve hiring success only a teensy-weensy bit … but, only if studies show it directly affects job performance. Aside from that? Nada!

Why Not?

Personality scores and KSAs are totally different things. I’ve done quite a bit of personality research and can assure you that personality scores and personal skills are two different things.  In fact, most all serious investigations shows that there is almost no relationship between scores on a personality test and KSAs (e.g., teamwork, conflict resolution, intelligence, learning ability, analyzing, planning skills, and so forth).

So now we have two pieces of trustworthy data: 1) personality scores have almost nothing to do with predicting job performance; and, 2) scores have almost no relationship with KSAs.

Why the Confusion?

One reason is semantics. Personality scores and skills often have the same name: i.e., analysis, teamwork, drive, and so forth. This makes it easy to confuse self-descriptions with skills. Another, as mentioned earlier, is the false belief that a self-description correlates with having a skill. And a third is caused by examining the wrong population.

You see, when people who are already working in a job all have enough skills to stay employed, small differences like personality tend to stand out; that is, job-group members are successfully employed and doing the same kind of work (i.e., a restricted-range population). Real proof, however, is obtained when you give everyone a personality test before hire, ignore the scores, and compare scores to performance six to nine months later. Unfortunately, all the vendors I know use restricted range populations and tell their clients their test predicts job success.  You can decide for yourself whether they are being intentionally deceitful or just don’t know any better.

What’s the Best Predictor?

If personality is not a good hiring predictor, what is? Tests, exercises, and simulations where the applicant has to demonstrate (not tell you about) job skills like being smart enough for the job, having the right interpersonal skills, having the right technical knowledge, and so forth.

Claims and Testimonials

Can you trust testimonials? ‘Fraid not. Smoke-and-mirrors vendors plaster testimonials all over the place. I think they believe their own hype. Always remember a free opinion is worth every penny you paid. Often their claims are enough to make one weep with either laughter or sorrow. Laughter, if you believe in claims like “90 percent effective,” “fake proof,” “totally reliable,” and so forth.  Sorrow, if you count all the qualified people who were rejected and all the wrong people hired based on bogus test scores.

Signal or Noise?

You might notice I always put test scores before job performance, and not the other way round. Clustering employees into high-low groups, giving them a personality test, and averaging scores is akin to throwing stuff on the wall and seeing what sticks. That’s because employees in the same job-group are often highly similar; performance ratings are not as simple as being “high-low”; group classification has more to do with opinions than facts; individuals in a group may not match their own group average; and, there is still no assurance personality caused performance.  If your current vendor used a high-low approach, Step 1 is to examine how well individuals in each high-low group actually matched their group average, and Step 2 is to find a new vendor because your old one doesn’t know what he/she is doing … better yet, start with Step 2.

Truth or Consequences

Does anyone really think applicants are totally honest? I know some vendors who claim their tests contain lie scales that identify fake responses, But as a professional test developer, I can say unequivocally that the only absolutely fake-proof test is one with no questions. No one can never know whether a test-taker is making him/herself look good, second-guessing a job-profile, or delusional.

Workshop Wows!

People often get “wowed” by scores in a workshop and assume it will work as a hiring tool. But how much wow magic does it take to add scores from five questions about being outgoing and social, and generating a report stating someone is “extraverted”?  Still wowed? Take a minute to Google the “Forer Effect” and read how people almost always agree with their personality reports. As a side note, I was once invited to take a vendor’s test for a spin. To test its accuracy, I chose answers randomly. The vendor proudly claimed the profile report fit me perfectly! I felt bad so I offered to help make the test better. It refused. Good marketer. Bad test developer.

On the Job Side

I’ll be the first to admit there are personality conflicts at work. But with whom? With today’s manager? With tomorrow ’s manager? With the work group? With the organization? Personality can certainly affect performance ratings once you are on the job, but it won’t predict skills beforehand.  They are, however, very good at producing clones — and cloning has a long history of contributing to organizational disasters.

For Information Only

Some users claim they use personality scores for information only, and not for decision making. Sure. Did we all just fall off the turnip truck? If test scores are information only, give the test afterhire, not before.

Guaranteed Results

I can guarantee with 100 percent assurance that anyone who intelligently tracks hiring results will soon discover which test is bogus and which one actually predicts job performance … then they can add up the wasted dollars spent on employing poor performers.

Conclusion

Personality tests, in even the best cases, are better left in training workshops where they can help people understand differences. Using them to make hiring decisions will always lead to turning away skilled candidates and hiring unskilled ones.

This article is provided for informational purposes only and is not intended to offer specific legal advice. You should consult your legal counsel regarding any threatened or pending litigation.

  1. Todd Lempicke

    Well stated! I’ve never met a personality test that I couldn’t manipulate with ease. Using these to make hiring decisions gives nice info and might be able to weed out crazies but shouldn’t be given much weight. A face-to-face meeting and conversation and interview reveals first hand information that’s more reliable.

  2. Scott Temby

    So, if this article is to be believed, how are companies able to use them in any evaluative capacity as part of a criteria for hiring, without the legal challenge of them being discriminatory and unrelated to the skills necessary to perform the job?

    I think many of us would like to see citations for the research either supporting or refuting the linkage to job success that is referenced in the article.

  3. Lauretta Russell

    …and the moon landings didn’t take place and this cell phone craze is probably going to die out. There are over 80,000 occupationally-related assessments, mostly personality. Imagine, none of those “work” and the 70+% of businesses using them in some degree are wasting their time and money. Within that universe of options, there are clearly huge differences in quality and effectiveness. However, to sweep the whole category into the dustbin, hints at somewhat less than thorough research on the subject. In fact, the connection between hard-wired personality traits and job performance is so well established that it scarcely warrants a raised eyebrow in the educated business world. For a more objective review of the assessment world, visit http://www.aboutassessments.com Chuck Russell, CEO BestWork DATA.

  4. Alan Roe

    I’m always amazed in business and personally how we all chase after the things we can’t have. The fountain of youth, celebrity, perfect children, immortality etc.
    If you alter H2O with anything, from the environment or directly you change the nature, behavior and integrity of the water. The same can be said with employees.
    The top tested, historical, evidence based salesperson in one organization can be a bum in another. Make employment like everything else, a day-to-day experience. Select based
    on impression and discard based on results. We select our food that way, if looks good we eat it but if it tastes bad we spit it out. If we are bad at picking out good food very quickly become under-nourished and die.
    The more reliance we put on the tools, the more validation we can put towards the inadequacy of the one or organization who uses those tools.

  5. Margot Nash

    While I agree that personality testing is often not related to skill level or ability, I’ve seen that the right assessment can be a fantastic predictor of job performance for several reasons:

    1. Job motivation is strongly related to job satisfaction, and job satisfaction stems from using one’s innate strengths at work. Therefore, a person whose job requires him/her to do the things that come naturally to them will ultimately be more motivated and perform better at the job than someone who is asked to do things that require stepping outside of what comes naturally to them. That’s not to say that the person who “bends” is unable or unwilling to do the job – they might even perform it quite well!) – It just means that they’ll enjoy it less in the long run.

    For example, many well-respected personality assessments include some form of a “detail orientation” scale. A person who scores low on this scale may be able to notice detail quite effectively on a job that requires it; But, if noticing details is a major part of their future job, it’s quite likely that they’ll lose interest or feel stressed out in the long term.

    For that reason, personality assessments should not be the “end all be all,” but when used with follow-up interview questions that ask candidates to elaborate on weaknesses, I know that they can be a very effective tool for predicting long term performance.

  6. Richard Araujo

    Trying to measure personality has nothing to do with whether or not a person will succeed in a job so much as it is whether or not they will succeed in a certain environment populated with certain people. Personality has little to no bearing on the job, it has everything to do with who a person works with and where. And while I agree every situation is fluid and those variables may change, not addressing them is probably a worse bet than at least attempting.

    I’d offer another reason why some testing might be helpful. It forces hiring managers to look at a candidate through a somewhat objectively defined framework instead of focusing on the one aspect of a candidate they either love or hate, and making a hire or no hire decision respectively with not enough consideration given to the whole candidate. While I’m not a fan of tests myself, I use them for this reason and this reason alone. There’s enough mystique around them that people actually do look at the results, and if I can use that to force at least a few hiring managers down a path to better hiring decisions then it’s worth it to me. I think you can accomplish the same thing yourself using a standardized scorecard, and I think we’ve all seen some. But those score cards require buy in from the managers that’s harder to get than with a third party vendor, for whatever reason. So I use the tests, and accept the fact that my job as a recruiter is to try and stop bad decisions as much as direct people toward good ones.

    I have also encountered aversion to skills testing specifically because they are more objective. There is significant resistance there because, let’s say you need an Excel expert, and you test a few people and hire the one candidate that got high to perfect scores all around. There’s no doubt, she knows Excel and can do the work. Yet, three months later the hiring manager is still claiming she, “isn’t working out.” This puts them in an awkward position because the obvious question is, “Why?” We know and have a pretty decent measure that says she has the skills, so what’s wrong?

    I’ve seen this situation before, and quite frankly it’s one of two things: a failure to properly define the work that needs to get done, which lead to bad hire; or a personality conflict that the manager isn’t competent enough to manage, or for whatever reason simply doesn’t want to. Maybe they don’t have to, being close friends with the company owner. Which leaves the company in an awkward position because the manager isn’t going to change themselves, certainly not over night and potentially never, so the ‘problem’ persists.

    Skills tests will always encounter resistance for those reasons. They require you adequately define the work that needs to be done so you can test correctly. Provided that has been done, they provide as objective of a measure as it’s possible to get that the person can do the work, which if coupled with a past history of success leads to uncomfortable questions for the manager that can not be brushed off or answered vaguely. They force accountability, which is why there is an aversion to them.

  7. Andrew D'Agostino

    Determining if personality is relevant to a job and accounts for a significant amount of variance in job performance for that job should be based upon a thorough job analysis. For many jobs, personality plays a role in determining job performance and success in that role. Personality assessments may not be effective for roles in which the situational variables on the job do not allow for the expression of behavior related to personality, however, for many jobs this is not the case.

    To say that skills and personality are not linked is perhaps a correct assertion (depending upon the skill of course), however, it is asserted that job performance is wholly accounted for based upon skills which is untrue. Skills certainly factor into job performance, but they remain only a part of the picture of job performance. In fact, KSA’s are mentioned in the article, to which “skills” are but a part of (Knowledge, skills, and abilities are different things).

    As Lauretta pointed out, sweeping personality inventories under the rug is perhaps unfair and will hurt the predictive power of a strong hiring process that incorporates tools such as these. Incremental predictability should be the goal of a robust hiring process and personality tests can and often do account for significant variance over and above other selection methods. This holds especially true for the personality trait Conscientiousness, which when measured correctly, has been known to add value over and above cognitive ability even (Hunter & Schmidt, 1998).

    Is a personality assessment a panacea for attaining top performers alone…most likely not and I would be wary of a vendor who does say this. Will it significantly help along with other selection methods to predict top performers pre-hire, yes it will and has shown this in the past (with reliable measurement and a strong criterion-related validation study).

  8. Jack Dermody

    “Using them to make hiring decisions will always lead to turning away skilled candidates and hiring unskilled ones.” That statement alone reveals an unscientific bias. Rare scientific statements contain that word “always”. In addition, you seem to be suggesting some/many/all employers give personality tests often to the exclusion of skill assessments and a battery of other assessment tools for hiring. To play the devil’s advocate, I feel placing so much emphasis on skills invites hiring people who lack social skills and who, in fact, may turn out to be toxic to teamwork and production. If some employers somewhere actually base decisions solely on personality tests, then shame on them. Smart employers perform due diligence and do their best to hire skillful people who are genuinely a good fit on the team. Thanks for listening.(Jack Dermody – Personality expert and author of Job Interviewers: How To Get Inside Their Heads).

  9. Kimberly Nassau

    I am not an expert on the subject, but I have several years of experience in using various assessments (always assessing before-hand, and evaluating results after months of use). I find this article to be so “definitive” on the subject as to hurt the credibility of the author. I am surprised that ERE published as-is. Thanks to all the other folks who made comments and provided additional, more balanced insight.

  10. Merlynn Bertini

    I am in agreement with Dr. Williams. Personality tests are exactly that–about personality. The tests definitely have merit, understanding communication styles,, effective team building, etc. But, in predicting success–no. Personality test have merit and effectiveness , but not in making hiring decisions.

  11. Roland Behm

    When people apply for a job online these days, they are increasingly being asked to take personality tests even before they exchange an e-mail or have a phone interview with a hiring manager.

    The problem is that these assessments have limited, if any, potential to predict job performance and are also being used to illegally screen out job seekers with mental disabilities.

    As a result, too many people living with mental disabilities that are willing and able to work remain unemployed or underemployed. Not only does the United States economy experience the indirect loss in productivity and tax revenue arising from the unemployment and underemployment of persons with mental disabilities, there is a direct, rising and material cost to the U.S. Treasury associated with income support payments, like Social Security Disability Insurance and Supplementary Support Income.

    The EEOC has been litigating the issue of personality testing and the ADA with Kroger and Kronos for more than five years. In addition, at least seven charges are currently pending with the EEOC, alleging that the personality tests are illegal medical examinations and illegally screen out persons with mental illness.

    See http://employmentassessment.blogspot.com/

  12. Keith Halperin

    @ Everybody: Here’s a paper which proves the point, though I’m not quite clear who’s point, or which one:

    http://psycnet.apa.org/journals/apl/82/1/30/

    In 3 prior meta-analyses, the relationship between the Big Five factors of personality and job criteria was investigated. However, these meta-analyses showed different findings. Furthermore, these reviews included studies carried out only in the United States and Canada. This study reports meta-analytic research on the same topic but with studies conducted in the European Community, which were not included in the prior reviews. The results indicate that Conscientiousness and Emotional Stability are valid predictors across job criteria and occupational groups. The remaining factors are valid only for some criteria and for some occupational groups. Extraversion was a predictor for 2 occupations, and Openness and Agreeableness were valid predictors of training proficiency. These findings are consistent with M. R. Barrick and M. K. Mount (see record 1991-22928-001) and L. M. Hough et al (see record 1991-06268-001). Implications of the results for future research and the practice of personnel selection are suggested. (PsycINFO Database Record (c) 2012 APA, all rights reserved)

    Cheers,

    Keith “I’ll Research It!” Halperin

  13. Roland Behm

    In a 2007 article titled, “Reconsidering the Use of Personality Tests in Employment Contexts”, co-authored by six current or former editors of psychological journals, Dr. Kevin Murphy, Professor of Psychology at Pennsylvania State University and Editor of the Journal of Applied Psychology (1996-2002), writes:

    “The problem with personality tests is … that the validity of personality measures as predictors of job performance is often disappointingly low. A couple of years ago, I heard a SIOP talk by Murray Barrick … He said, “If you took all the … [five factors], measured well, you corrected for everything using the most optimistic corrections you could possibly get, you could account for about 15% of the variance in performance [between projected and actual performance].” You are saying that if you take normal personality tests, putting everything together in an optimal fashion and being as optimistic as possible, you’ll leave 85% of the variance unaccounted for. The argument for using personality tests to predict performance does not strike me as convincing in the first place.”

    As stated in the article:

    “The journal editors were used for … two reasons. First, they have reviewed and made publication decisions on over 7,000 manuscripts …; hence, they are in a good position to judge the cumulative personality research. Second, although these editors are among the most widely published authors in the field of I-O psychology …, none of them have a primary research stream on personality testing or could be considered an advocate or a critic of personality testing in the published literature. Therefore, they are relatively impartial.”

    http://onlinelibrary.wiley.com/doi/10.1111/j.1744-6570.2007.00089.x/abstract

  14. Jack Dermody

    If you focus unduely on some kind of correlation between personality measures and performance, you might miss the true value of personallity assessments. A temperament sorter will say very little about computer programming skills, but it WILL fairly accurately suggest how the programmer will interact with bosses and colleagues. The sorter will predict leadership and thought processes, e.g., strategic, logistical, tactical or diplomatic. A sorter will predict fairly accurately whether you are hiring a “people person” or someone who is more “task oriented.” I am sure most of us can name colleagues from hell whose negative personalities overpowered the value of “excellent performance skills” and that a variety of high quality assessments might have prevented such a train wreck.

  15. Roland Behm

    Under the ADA, a personality assessment being used, in whole or in part, for hiring decisions, must not be an illegal pre-employment medical examination and it must not screen out persons with mental illness. If it is an illegal pre-employment medical examination, then all job applicants – not just those with mental illnesses – are potential plaintiffs.

    There has been no public disclosure of any studies addressing the potential adverse impact of FFM-based personality assessments on persons with mental disabilities. One prominent assessment company, Kronos, has claimed in a court document, that there is “no known method … to ascertain adverse impact against the entire generic category of disabilities.”

    As stated in a 2004 article published by the the Journal of Rehabilitation Administration, “studies of sub-groups, such as individuals with mental illnesses or cognitive impairments could be conducted to determine the potential, and perhaps likelihood for, pre-employment test results unfairly penalizing these individuals in the employee selection and hiring stages …”

    Kronos’ “inability” to examine the impact of personality assessments – or its unwillingness to disclose its potential discriminatory impact – on the hiring of persons with mental illness may be the reason Kronos has spent the last 5+ years (unsuccessfully) contesting the EEOC’s subpoenas for such information. http://www.law360.com/articles/378514/3rd-circ-says-eeoc-can-broaden-probe-in-kronos-bias-case

    It is not the assessment company (Kronos, in this case) that will be liable for any potential damages, it will be the employer (Kroger, in this case). The ADA allows an employer to use a third party to administer assessments, but responsibility for any liability remains with the employer.

    As to the accuracy of personality tests as temperament sorters, a May 2013 article in Fortune magazine examines the MBTI reads:

    The interesting — and somewhat alarming — fact about the MBTI is that, despite its popularity, it has been subject to sustained criticism by professional psychologists for over three decades. One problem is that it displays what statisticians call low “test-retest reliability.” So if you retake the test after only a five-week gap, there’s around a 50% chance that you will fall into a different personality category compared to the first time you took the test.

    A second criticism is that the MBTI mistakenly assumes that personality falls into mutually exclusive categories. You are either an extrovert or an introvert, but never a mix of the two. Yet most people fall somewhere in the middle. If the MBTI also measured height, you would be classified as either tall or short, even though the majority of people are within a band of medium height.

    The consequence is that the scores of two people labelled “introvert” and “extrovert” may be almost exactly the same, but they could be placed into different categories since they fall on either side of an imaginary dividing line.

    http://management.fortune.cnn.com/2013/05/15/myers-briggs-problems/

  16. Erin Cox

    I recently completed a validation study of the abilities and personality tests that we use at the company I work for (a multinational with 125 000 + employees).

    The strongest positive correlation was found to be between two of the personality test dimensions and the performance review rating. The abilities test results on the other hand were found to have a weak, negative correlation with the performance review ratings.

    I found this surprising given most academic articles on the subject I have read state that abilities tests possess the strongest validity. I can’t argue however with data from my own organisation that I have collated myself.

    The applicants had all completed engineering degrees so one could confidently assume they all possess a reasonable level of intelligence. However if you are recruiting candidates that possess an advanced degree as a minimum des this then render abilities testing less valid as a means to differentiate between a pool of applicants? I am interested to hear others opinions.

  17. Richard Araujo

    Erin,

    It’s possible their engineering knowledge wasn’t the deciding factor in performance. That’s what I meant in my comment when I mentioned defining the work that actually needed to be done. There has to be a certain level of intelligence to get an engineering degree, but the job isn’t getting an engineering degree. That was the ‘job’ in college, now the job is really design of PCBs, or product management, or managing external design, or testing for compliance with FCC and UL and the like, etc. Having the degree doesn’t prove you can do any of those things.

  18. Jack Dermody

    Getting back to the role of personality assessments, here’s an example. I will assume you understand Myers-Briggs. If a job applicant for civil engineering happened to test out as an ISTJ (called “the inspector” by some temperament experts), I would expect and reasonably hope the applicant would be a nose-to-the-grindstone, no-nonsense, focused engineer. Actual knowledge and skills would have to be assessed separately, but an ISTJ tends to be a good bet for certain engineering positions. On the other hand, if the applicant were an ENFP (known sometimes as “the champion”), I would worry such a person might not be focused enough, might be too people-oriented and, thus, I would take extra lengths to interview thoroughly, call previous employers, test for actual skills, etc. Then here is the exciting news: if an ENFP happens to be an ace engineer, I might go out of my way to hire him/her because the team could probably use the good communication and natural positivity of an ENFP. My point here is that personality assessments offer information that so-called skills tests and academic records cannot deliver completely.

  19. Casey Brady

    Ditto this article! We used personality tests and I hated them. It would weed out top talent. Thankfully the recruiters/hiring managers rallied together (collected data supporting our claims) and we pushed this up the ladder to get it out of our interview process.

  20. Keith Halperin

    @ Jack, and others Re: MBTI (http://en.wikipedia.org/wiki/Mbti
    Reliability

    The MBTI is “one of the most popular personality questionnaires”.[53] Some researchers have interpreted the reliability of the test as being low. Studies have found that between 39% and 76% of those tested fall into different types upon retesting some weeks or years later.[37][39]

    One study reports that the MBTI dichotomies exhibit good split-half reliability; however, the dichotomy scores are distributed in a bell curve, and the overall type allocations are less reliable. Also, test-retest reliability is sensitive to the time between tests. Within each dichotomy scale, as measured on Form G, about 83% of categorizations remain the same when individuals are retested within nine months, and around 75% when individuals are retested after nine months. About 50% of people tested within nine months remain the same overall type, and 36% remain the same type after more than nine months.[54] For Form M (the most current form of the MBTI instrument), the MBTI Manual reports that these scores are higher (p. 163, Table 8.6).

    In one study, when people were asked to compare their preferred type to that assigned by the MBTI assessment, only half of people picked the same profile.[55] Critics also argue that the MBTI lacks falsifiability, which can cause confirmation bias in the interpretation of results.

    A number of scholars argue that criticisms regarding the MBTI mostly come down to questions regarding the validity of its origins, not questions regarding the validity of the MBTI’s usefulness.[56] Others argue that the MBTI can be a reliable measurement of personality; it just so happens that “like all measures, the MBTI yields scores that are dependent on sample characteristics and testing conditions”.[57]
    Utility

    In her research, Isabel Myers found that the proportion of different personality types varied by choice of career or course of study.[1]:40-51[9] However, some researchers examining the proportions of each type within varying professions report that the proportion of MBTI types within each occupation is close to that within a random sample of the population.[37] Some researchers have expressed reservations about the relevance of type to job satisfaction, as well as concerns about the potential misuse of the instrument in labeling individuals.[37][58]

    CPP became the exclusive publisher of the Myers-Briggs instrument in 1975. They call it “the world’s most widely used personality assessment”, with as many as two million assessments administered annually.[59] CPP and other proponents state that the indicator meets or exceeds the reliability of other psychological instruments and cite reports of individual behavior.[42][60][61]

    Some studies have found strong support for construct validity, internal consistency, and test-retest reliability, although variation was observed.[43][44] However, some academic psychologists have criticized the MBTI instrument, claiming that it “lacks convincing validity data”,[33][38][39][62] while some studies have shown the statistical validity and reliability to be low.[37][39][63]

    Studies suggest that the MBTI is not a useful predictor of job performance.[37][41][64] As noted above under Precepts and ethics, the MBTI measures preference, not ability. The use of the MBTI as a predictor of job success is expressly discouraged in the Manual.[7]:78 However, the MBTI continues to be popular because many people are qualified to administer it, it is not difficult to understand, and there are many supporting books, websites and other useful sources which are readily available to the general public.[65]

    ……………………………………..

    Meanwhile, here’s a kind of test I think probably SHOULD be administered more often (and to existing employees,too):

    http://helloquizzy.okcupid.com/tests/the-are-you-a-sociopath-test
    http://www.allthetests.com/quiz18/quiz/1140475124/Are-You-A-Sociopath
    http://www.vodahost.com/web-hosting-sociopath-test-sociopath-definition.html/

    -Keith “ENFP, But What Difference Does It Make?” Halperin

  21. Jack Dermody

    @Keith Thank you for jumping in with some research figures. Your style and the suggested links make it hard to tell when you are speaking tongue-in-cheek … and when otherwise. In any case, assuming you want to maintain high ground, I trudge on. I tell my audiences that personality assessments — even the best ones — are accurate no more than 80% of the time for 80% of the people. (Do the math.) Because we are dealing with human beings, tools like MBTI are more like a mother’s hand assessing a fever than a sophisticated digital thermometer measuring accurately. All people who do these assessments (and they usually are SELF-assessments, by the way) are wise to be skeptical. However, when they get the results, they want to ask themselves what might actually true about the results, then ask themselves how this information might help. The same for employers: “Okay, Keith says he’s an ENFP. He himself is asking what difference that makes. If we look at ENFPs on a normal curve, it will be highly unlikely that he will be chosen to train to be a sniper in a platoon, unlikely he’ll enjoy keeping books for a logistically-oriented company like UPS or a retail store; on the other hand, he might be a terrific diplomat and idea person based in the executive suite or HR department of a large company or organization, or he might like to be part of a recruitment or marketing team that finds people to join a cause he deeply believes in. At the very least, I’ll bet he’s a fun and interesting friend to have.” (All this just for starters, Keith…let me know if you think this is crazy.)

    When I find out that somebody is an NTJ or STF, of course I will want to get to know them before deciding how close to them I want to get. HOWEVER, the odds are very good that I will profoundly irritate them sooner or later. They tend to shut me out or off just because I don’t seem serious enough or responsible enough in their eyes. I suppose the problem is that I choose what I want to get serious about, then speak of other things perhaps too lightheartedly. BTW, Keith, I too am an ENFP.

    Back to the issue at hand. I am skeptical of all soft assessments…and we should all be. I am also skeptical of so-called hard-skill assessments. But I know for certain that a careful look at a well-thought-out combination of experience, education, skills assessments, personality assessments and interviews will deliver the top talent that a company desires.

  22. Keith Halperin

    Thanks Jack. I’ve found that when I make a serious, fact-supported point people think I’m joking, and when I make the most outlandish and implausible remark they think I’m serious, so I let people take what I say as they wish, with a grain of salt (unless they’re watching their blood pressure).

    I used to be very much into MBTI and Keirsey, etc. but now I believe that MBTI is very unscientific, and I lean more toward “Big 5″. (http://en.wikipedia.or /wiki/Big_Five_personality_traits) as being less unscientific. However: Big Five and work success
    Controversy exists as to whether or not the Big 5 personality traits are correlated with success in the workplace.

    The Big-Five Inventory can be administered by employers to job applicants. It is believed that the Big-Five traits are predictive of future performance outcomes. Job outcome measures include: job and training proficiency and personnel data.[98] Research demonstrating such prediction has been criticized, in part because of the apparently low correlation coefficients characterizing the relationship between personality and job performance. In a 2007 article[99] co-authored by six current or former editors of psychological journals, Dr. Kevin Murphy, Professor of Psychology at Pennsylvania State University and Editor of the Journal of Applied Psychology (1996-2002), states:

    A validation study in 1992, conducted by Paul Sinclair and Steve Barrow, involved 202 Branch Managers from the then TSB Bank. It found several significant correlations with job performance across 3 of the Big Five scales. The correlations ranged from 0.21 to 0.33 and were found across three scales: High extraversion, low neuroticism and high openness to experience.[100]

    The problem with personality tests is … that the validity of personality measures as predictors of job performance is often disappointingly low. A couple of years ago, I heard a SIOP talk by Murray Barrick … He said, “If you took all the … [five factors], measured well, you corrected for everything using the most optimistic corrections you could possibly get, you could account for about 15% of the variance in performance [between projected and actual performance].” … You are saying that if you take normal personality tests, putting everything together in an optimal fashion and being as optimistic as possible, you’ll leave 85% of the variance unaccounted for. The argument for using personality tests to predict performance does not strike me as convincing in the first place.

    Such criticisms were also put forward by Walter Mischel[101] () whose publication caused a two-decades’ long crisis in personality psychometrics. However, later work demonstrated (1) that the correlations obtained by psychometric personality researchers were actually very respectable by comparative standards,[102] and (2) that the economic value of even incremental increases in prediction accuracy was exceptionally large, given the vast difference in performance by those who occupy complex job positions.[103]

    There have also been studies that link national innovation to openness to experience and conscientiousness. Those who express these traits have showed leadership and beneficial ideas towards the country of origin.[104]

    Some businesses, organizations, and interviewees assess individuals based on their 5 personality traits. Research has suggested that individuals who are considered *leaders typically exhibit lower amounts of neurotic traits, maintain higher levels of openness (envisioning success), balanced levels of conscientiousness (well-organized), and balanced levels of extraversion (outgoing, but not excessive).[105] Further studies have linked professional burnout with neuroticism, while linking extraversion to enduring positive work experience.[106] When it comes to making money, research has suggested that those who are high in agreeableness (especially men) are not as successful in accumulating income. It is possible that these individuals are too passive and do not aspire to obtain higher levels of income.[107]

    Studies have utilized big-five personality inventory in college students to determine that hope, which is linked to agreeableness has a positive effect of psychological well being. Individuals high in neurotic tendencies are less likely to display hopeful tendencies and are negatively associated with well-being.[108] Personality can sometimes be flexible and measuring the big five personality for individuals as they enter certain stages of life may predict their educational identity. Recent studies have suggested the likelihood of an individual’s personality affecting their educational identity.[109]

    There have also been studies that link use of social media to the five traits. On Facebook, the predictor for number of contacts is also the predictor of friends in the real world (extraversion).[110] On Twitter, both popular users and influentials are extraverts and emotionally stable (low in the trait neuroticism). Popular users are also found to be ‘imaginative’ (high in openness), while influentials tend to be ‘organized’ (high in conscientiousness).[111]

    …………………………………………

    In lieu of personality tests, why not use the “likeability question component” for F2F interviews? It’s highly subjective, but still going to be present anyway, so might as well use it.

    “When I find out that somebody is an NTJ or STF, of course I will want to get to know them before deciding how close to them I want to get. HOWEVER, the odds are very good that I will profoundly irritate them sooner or later.” …

    You might not have guessed it, but I’ve been known to profoundly irritate people on occasion, too. I don’t think it’s because I’m an ENFP, but because **I’M A PROFOUNDLY IRRITATING PERSON!

    Cheers,

    Keith

    * But see: http://www.forbes.com/sites/victorlipman/2013/04/25/the-disturbing-link-between-psychopathy-and-leadership/,

    http://www.thebusinessowner.com/business-guidance/professional-development/2009/11/the-sociopath-next-door,

    http://keenetrial.com/blog/2011/08/01/not-all-psychopaths-are-in-prison-some-are-in-the-boardroom/

    **Think I know why, too…

  23. Jack Dermody

    @Keith and this whole totally engaged gang. This discussion is more than worthwhile. Thanks. My thoughts based on Keith’s ideas and data: If I had to give a number to a decent weight to even the most respected personality assessments, e.g., the Big 5, I’d say 15% in the hiring process. Sensitive applicants might justifiably sue a company if they found out a personality test counted for much more than that.

    But let’s not discount factors that can make a difference on teams. Super high Agreeableness might not be a good thing for a sales position requiring cutthroat competiveness. Excessive Openness could get a platoon leader’s people killed in a skirmish.

    Let me give you a real world example. I get called in to help teams where relationships have been toxic. Believe it or not, people who work in IT experience quite a bit of this. It’s not unusual for the majority on an IT team to be introverted, independent, uncomfortable with small talk, task-oriented and time-conscious. You might guess correctly that their diplomatic communication skills can be lacking.

    When such a team has a chance to hire new people, you can’t fault them for looking for talent with communication skills. Simply in the interview process, it’s pretty easy to dig out traits with carefully worded questions, e.g., “Thinking of all the managers you have every had, who communicated with you best and how would you describe that communication? Describe a time when you had a fundamental disagreement with a boss or colleague and tell us how the disagreement was resolved. Describe a work environment for which you would leap out of bed in the morning. What does ideal teamwork look like to you?”

    One of the advantages of having a personality assessor on the hiring team is that it’s possible to spell out some quantifiable and defensible descriptors for a new hire — that can provide a quantifiable and defensible “good fit” matrix.

    And the IT example is a good one to point out that Big 5 would provide more info than MBTI. Why? Probably the majority of IT applicants could be INTJs and INTPs (like the rest of the team), so the additional factors provided by Big 5 would fill in gaps. And for “curing” a toxic environment, thorough background checks are more important than ever.

    Keith: I mistyped (intended pun) in a paragraph. The types were supposed to be NTJs and STJs who are my natural “nemeses”. For me, the beauty of obsessing about personality typing is that I find fewer reasons to let people upset me because people do belong to a billion-person block of types just like them who happen to be pretty much hard-wired, so why do I need to lose sleep over them?

  24. Roland Behm

    Jack,

    You state that “[s]ensitive applicants might justifiably sue a company if they found out a personality test counted for much more than” 15% of the hiring decision.

    The challenge is that personality tests are being used to make 100% of the decisions at the initial stage of tens of millions of hiring decisions.

    Most of the large retail companies, think restaurants, pharmacies and home improvement stores, require an applicant to take an online personality tests even prior to submitting their resumes. These tests are immediately scored, usually with the colors green, yellow and red. Applicants scoring green move forward in the hiring process – potentially to an interview and an offer. Applicants scoring red are excluded from participation in the hiring process.

    These personality tests, think Kenexa, Kronos, etc, are developed using the five-factor model of personality. The EEOC is currently challenging Kroger’s use of the Kronos assessment on the basis of its potential discriminatory impact on persons with mental illnesses.

    According to a statement filed by Kronos in the litigation, determining whether a test discriminates against persons with disabilities (e.g., mental illness) is impossible.

    As stated in a journal article several years ago:

    Job applicants are seldom given the underlying reasons for non-hiring. The dearth of litigation involving employment tests of individuals with disabilities does not, in and of itself, prove that these tests do not contribute to discrimination in hiring. Rather, it is more accurately a reflection of the practice of companies marketing employment tests advising employers not to disclose the impact of the test scores on their hiring decisions. The widespread adoption of these non-disclosure policies by employers effectively forestalls legal challenges to the lawfulness of pre-employment testing, as most rejected applicants will lack knowledge of the significant impact their test results had in the hiring decision. Consequently, employment tests have been insulated from judicial scrutiny.

    Ilana S. Lehmann, William Crimando, Veiled barriers: Pre-employment Testing and Individuals with Disabilities, Journal of Rehabilitation Administration 01/2006; 28:11-22.

    Contrary to Kronos’s statement, it is possible to study whether personality tests discriminate against persons with mental illness. Otherwise, how would pharmaceutical companies be able to get FDA approval for medicines that treat mental illnesses.

  25. Keith Halperin

    @ Jack. Thanks again.
    A few points: I believe successful job experience counts personality assessment. Example: I wouldn’t give a candidate for an experienced sales position a Big 5 Personality Test, I’d ask them first to show me their W2…
    On a side note re: your IT teams- I’m not clear if you’re implying that the teams are toxic because of the large prevalence of INTJs (introverted, independent, uncomfortable with small talk, task-oriented and time-conscious.)This is not only a personality type of many IT people, it’s also characteristic of many people on the Autistic Spectrum. Perhaps instead of trying to label this environment as toxic and look for people who aren’t like this, perhaps it would be better to work to create an environment which could function well with people who don’t need to communicate like this. Let me give a related example: many of the trendy “open, collaborative environments” with everybody cheek-by-jowl are very unpleasant for just the people you’ve mentioned. (I can’t imagine an INTJ [whether on or off the spectrum] coming up with something like this). A major point to consider: the less you need to have people in constant F2F contact with each other (which fits introverts anyway), the less you need to pay attention to personality fit: yet another good argument for remote work for those who can do it successfully.

    I also like to be able to “size people up”- it helps me to deal with them more effectively. That’s why I think it’ll be cool where in a few years we’ll be able to use our smart phones or smart glasses and quickly come up with a person’s digital dossier linked to their biometric data.

    @ Roland: Very interesting. ISTM that these personality tests seem to be concentrated on lower-level and lower-paid employees. Do lots of directors. VPs, etc. end up taking them, too?

    Cheers,

    Keith

  26. Jack Dermody

    @Roland I’m learning so much from you guys. I have to say I’ve dealt with Kroger employees in several far-flung states and have to report that the level of conscientiousness and customer savvy feels pretty good to me. So one wonders how much of that comes from the hiring process, training, and on-going management.
    Still, I worry about organizations who place too much faith in psychological assessments and risk missing out on terrific employees and, worse, depriving worthy people of a livelihood. Many years ago, I was pre-assessed as “difficult to live with” but ended up the favorite to live with by all 150 trainees. The assessor called me into his office and screamed, “You fooled them all!” LOL The truth is simple: I had excited a dark place in my life, found the new employer, and quickly became ecstatically happy with their goals, people, energy, and core activities. How about THEM apples!

    @Keith No, any team at all can live in a toxic environment. I get called into places where conflict is high, change management is taking place or in the offing — i.e., difficult, toxic, stagnant, needing outside help. Keith, I appreciate your thought process and the questions you ask.

  27. Erin Cox

    Richard,

    The applicants weren’t tested on their engineering knowledge. They were tested across three dimensions – verbal reasoning, numerical reasoning and inductive reasoning. There was a very weak negative correlation between the abilities testing scores and the performance ratings of the applicants once they became employees.

    On the other hand there was a strong positive correlation between two of the personality assessment dimensions and the performance ratings of the applicants once they became employees.

    This to me indicates that not all organisations are the same and not all roles are the same when it comes to the validity of applicant testing tools and sweeping statements can’t be made in relation to the validity of abilities testing over personality testing.

    I work for a particularly large, global organisation (our internal recruitment team has over 40 team members recruiting for just one division) so I have access to large volumes of both selection and performance assessment data. It mnakes for interesting analysis!

  28. Keith Halperin

    @ Jack: “difficult, toxic, stagnant, needing outside help”
    Welcome to my world!

    ” Keith, I appreciate your thought process.”
    So now you’re accusing me of “thinking”? Will this abuse never end? At least you’ve not yet called me “a thought leader”- that would be totally reprehensible….

    Cheers,

    Keith “Laughing All the Way to the Poor House” Halperin

  29. Erin Cox

    For those interested, the personality testing we use is the Occupational Personality Questionnaire (OPQ) and the two dimensions that had the strongest positive correlation (both >0.53 and far above any other assessment measure we use including abilities testing and GPA results) were:

    Communicates with Impact
    Builds & Maintains Positive Relationships

  30. Roland Behm

    @Jack wrote, “Still, I worry about organizations who place too much faith in psychological assessments and risk missing out on terrific employees and, worse, depriving worthy people of a livelihood.”

    Exactly right. The use of personality tests to screen the initial applicant pool eliminates from employment considerations many persons who are ready, willing and able to work and capable of doing the fundamental tasks of the job.

    Not only do they lose the opportunity to earn a livelihood, they also miss out on the structure and socialization benefits provided by work.

    For society, the knock-on effects of this heavy-handed (and illegal) screening out process are significant. Persons with mental illness, whether diagnosed or undiagnosed, are eliminated from employment consideration. As a consequence the pool of persons unemployed who have a mental illness has been disproportionately increased over the last 15 years.

    These persons are then eligible to apply for income support payments like Social Security Disability Insurance and Social Security Insurance. Over the last 15 years there has been a material increase in both the absolute number and percentage of persons with mental illness receiving income support payments – monthly SSDI award payments are currently in excess of $10 billion/month.

    People with mental illnesses constitute the largest and most rapidly growing subgroup of Social Security disability beneficiaries. In 2011, 31 percent of persons receiving SSDI had a mental illness (not including claims relating to intellectual disability). Some persons contend that rising disability awards for mental illness reflect a “broken” system that provides benefits to those who should not receive them; others point out that income support makes it easier for persons with mental illness to live in the community.

    There is an alternative. The income support programs like SSDI may be working as designed, but those programs did not anticipate the impact of the widespread use of pre-employment personality assessments and the resulting material increase in the absolute number and percentage of unemployed persons with mental illnessses seeking SSDI and SSI benefits as a consequence of the use of those assessments.

    The failure of the testing companies and employers to use a screening process that does not discriminate against persons with mental illness – people who have the skills and desire to work – results in significant costs, both to the affected individual and to U.S. taxpayers.

  31. Jim Blair

    This author pans personality tests many of which are used for hiring decisions. MBTI has already been mentioned. I agree it should not be used for hiring. It is by its very construct an ipsative (self measuring) instrument. That is, the assessment taker is describing themselves. The best assessments are normative (measured against others) and values attained are usually described using a ‘bell curve’ I have used the MBTI for team building and development purposes and am careful to describe MBTI’s limitations. Just because all your engineers are ISTJ’s doesn’t mean a degreed engineer who is not an ISTJ will be unsuccessful at you company either as an engineer, manager or anything else.

    The DOL has information regarding assessments and tests on their website: http://wdr.doleta.gov/opr/fulltext/99-testassess.pdf.
    An adequate test reliability coefficient and value interpretation:
    .90 and up: excellent
    .80 – .89 good
    .70 – 79 adequate
    < .70 may have limited applicability

    Erin, why are you using anything less? A .53 is no better than tossing a coin .50; worse, it wastes employee’s time, company’s money, causes the probability of missing good candidates and subjects you to an indefensible lawsuit.

    To use an assessment properly, one should assess their best employees in a given position. A solid assessment will identify how these individuals think; natural tendencies, behaviors, preferences and attitudes toward numerous key performance issues. Once these dimensions are known it’s possible to use them to identify candidates who are likely to be successful in that particular position. However, an assessment is only one third of the recruiting process; one must use skilled trained interviewers and a rigorous background check.

    What concerns me is those readers who only see pre-employment tests and falsely assumes all tests are useless. I believe the author did a disservice to those employers using quality assessments properly.

    Keith…sensationalism sells, makes for good headlines but a poor article. This is equal to those in Yahoo which I no longer read.

    If I was the judge in the cases described: guilty. Erin, if I was the judge adjudicating a case against your company because of that assessment; summary judgment for the plaintiff.

  32. Erin Cox

    Jim,

    Out of interest, have you completed a validity study yourself of assessment instruments used at your organisation?

    Interested to know.

  33. Jack Dermody

    Erin, Jim et al, as much as I flog the terrific value of personality studies and their application to the workplace, I remain doubtful about high levels of personality assessment reliability and validity — no matter how sophisticated the designers claim their own to be. If an organization indeed wants defensible recruiting and hiring, I’d say focus on the the experience and skills of all folks in the hiring chain — from resume readers to interviewers and decision makers. Straight-A people. Yes, use the best tools. Yes, do the research that Jim suggests. In the end, I see no evidence that finding and hiring the right employees is anything less than a best-educated mix of experience and science and art. People are complicated. The difference between your view of the person you hired and that same person two years down the road can be startling. Loud-mouthed mavericks may turn your company into a major contender. Introverts can solve people problems. “Perfect” people at at an interview can just as well perfectly turn today’s harmonic relationships into a toxic hell in a matter of weeks or months.

  34. Roland Behm

    The challenge for many companies is that they do not put their money where their mouth’s are. Most companies make some sort of claim along the lines that “employees are our most valuable assets;” but who treat HR (recruiting, hiring, development) as a necessary evil, at best.

    HR tends to be viewed as a cost center that adds nothing directly to the company’s performance (i.e., revenue, profitability, market share). Consequently, HR departments tend to to be funded at “life support” levels – asked to do more and more with less and less.

    Take for example the rise in pre-employment personality testing by many of the largest employees. I would suggest that the primary motivating factor behind the use of those tests is to try and tame the river of applicants for each position arising as a result of the growth in Internet job postings over the past 15-20 years. It is not to find the best or most appropriate fit; rather, it is a way to eliminate the most applicants with the lowest cost – both direct costs and potential costs associated with violating employment discrimination laws.

    Using a recognized testing company provides some “safety in numbers” comfort – i.e., nobody ever got fired for choosing IBM – but has created substantial and systemic risk. If one employer who uses a testing company’s test is challenged and loses in court, all companies who use the company’s tests are at risk.

    That risk could be ameliorated, to some extent, if the companies using the tests engaged in appropriate oversight, review and validation, either on their own (if they have the experience in-house) or through the use of independent third parties. An HR group seeking to undertake that oversight, review and validation will be confronted with the challenge of explaining why the company needs to spend more money to “oversee” a testing company that HR has told management is the key to meeting HR’s KPIs like time to hire, cost per hire, etc.

    Until testing litigation takes off along the lines of FLSA litigation and results in significant damages and a radically different risk profile, it will be hard for HR to get funding to keep the company safe. But by the time that occurs, it may be too late for some employers and testing companies.

  35. Dr. Wendell Williams, MBA, Ph.D

    Personality test articles seem to bring out both the best and worst of readers…and this was no exception..

    First, the best: kudos to all those people who are taking the time to learn more about personality and it’s relationship to performance. You are providing some great examples of test professionalism.

    Next, shame on all those who vigorously attack anyone who challenges their sacred cows. I do not make this stuff up…it has been a topic of thousands of personality researchers for decades… yes decades… and their general conclusions(although unprofessional vendors hate it) is that personality is a lousy predictor of job skills.

    If you are vendor who does not like this kind of criticism, please share with me your test development manual. I’ll examine it using the 1999 Standards for Educational and Psychological Testing and write up the results in this forum (anyone want to bet how many vendors will participate?)

    Otherwise, I’ve noticed many of the wrong-headed opinions voiced on this topic seem to come from: 1) people who confuse skill-prediction with employee-performance (i.e., applicant skills are very different, but employee skills are very similar); 2) those who don’t understand the difference between correlation and probability(i.e. correlation compares sets of historical data, while probability predicts success); and, 3) believing the claims of vendors who failed to follow professional test development standards.

    In case you are using ANY common workshop test to hire people, you really need to look deeper at their theory (i.e., does it have ANYTHING to do with job performance), validity (i.e., what do hi-lo scores have to do with hi-lo job performance), and stability (i.e., are the scores stabile)..until then, keep them in the workshop.

  36. Roland Behm

    It’s not just confusing correlation and probability, it’s confusing correlation with causation. Causation means X causes Y; correlation means X and Y tend to appear together.

    For example, in the first half of the 20th century it was thought that ice cream caused polio. Why? Because cases of polio spiked during the summer months and so did ice cream consumption. See the Freakanomics video on causation vs. correlation: http://www.youtube.com/watch?v=lbODqslc4Tg

  37. Jack Dermody

    Dr. Williams’ latest comments are well-taken. As a vendor of “workshop assessments,” I will say YES that I’d be horrified if an employer used mine to hire people. If the employer is not willing to dig deep and thoroughly vet an applicant, it deserves the disappointing people it hires. As an aside, I wrote an entire book for job applicants based on understanding the values and meeting the needs of the four classic temperament types. It’s really a tool to help job applicants prepare more thoroughly than they might otherwise. If you are on THIS discussion, I would love to gift you a copy and would appreciate your appraisal. Send me an email at dermody@cox.net with your email address for a Kindle version; a mailing address for a paperback version. Check it out, if you want, at Amazon — Job Interviewers: Get Inside Their Heads.

Post a comment

Please log in to post a comment.

Note: You need to sign up for an account on our new commenting system if you haven't already done so — even if you have an existing ERE account. Find out why »

Login Information