One of my clients, an accomplished CFO involved with global M&A responsibilities, wanted to make sure that his new finance hires could really use all the brainpower they had been blessed with, so he incorporated a critical thinking test as part of the interview process. He was pleased with the results, seeing an increase in the problem solving abilities of his new team members. However, when he decided to take the instrument himself, there was a different reaction …
“I don’t understand why I only scored in the fiftieth percentile!” he thundered.
I pointed out that that is well within the norm for senior managers. “There must be something wrong with the test,” he continued.
“You and I examined the reliability and validity coefficients together,” I said.
“I wasn’t as focused as I should have been when I took it,” he implored.
“Would you like to take the instrument again?” I offered.
(after a long pause) he said, “Well, no, I guess not. It really is a bugger, isn’t it?”
“A lot of folks say that,” I said. “And, remember, you now have people working for you whose processors can scream at warp speed.”
He smiled.
The #1 Sought After Skill
Critical thinking has been rated the #1 desired skill in key contributors and senior level leaders, according to surveys conducted by organizations such as SHRM and AMA. And, as Socrates understood, although it can be learned, organizations today don’t have the luxury of teaching this skill. They need people already adept at:
- Accurately understanding problems,
- Analyzing evidence, and
- Making good decisions.
With 7-10,000 baby boomers retiring every day, the need for critical thinkers has never been greater.
Hiring High Potentials or High Performers?
Yes, please.
During my years as a retained search consultant in the golden days of the late 1990s, companies had more room for choosing both types of candidates — those who could make a significant impact now and those who were solid, but were oozing with potential to be developed. Today, the market demands both.
So, how do we determine critical thinking ability in candidates?
To Test or Not to Test?
There are a few good instruments out there, including Pearson’s Watson-Glaser II. However, many search consultants I know choose to develop their own critical thinking interview questions. Both are valid methodologies.
And, what about combining the two strategies? After you have culled down the list of candidates to the top three to five, give each a high-quality, critical-thinking assessment. Then, incorporate the results into your last round of interviewing, developing questions specifically targeted to the possible weak links identified by the testing. I believe this to be the most powerful approach.
Drill Down from Targeted Questions
So, what types of questions really get at critical thinking ability? Behavioral questions can be used, but be careful — remember that candidates will tend to showcase war stories/accomplishments which only show their best. Motivation questions, while trendy and powerful, are as much about heart as head — while they are mandatory for any 2012 interview, they don’t really get at critical thinking directly. Situational questions can also provide us with a respectable platform from which to examine a candidate’s critical thinking ability.
And, once we have developed the right question for the right candidate, the next key is following up their answer with drill-down questions.
Here are a few sample questions and follow ups you might build out from:
Behavioral Questions
Describe a complex situation in which you …
… had to think through information which conflicted with your own viewpoint or beliefs.
… had to make a critical choice based on incomplete data or inputs.
… convince others to examine different approaches surrounding a contentious topic.
Behavioral Drill-down/Follow-up Questions
List your basic assumptions when you first considered the situation.
What were your individual actions?
How did you determine which actions to take?
Please describe all the intended and unintended outcomes.
What could you have done differently to achieve similar outcomes?
What could you have done better?
Situational Questions
A cross-functional (you pick the disciplines) team you are serving on is tasked with identifying, analyzing, and reporting on operational efficiencies. The efficiency data provided to the team by senior management is accepted by all team members as accurate, but you recognize it as faulty. Describe, in detail, how you would proceed.
A strong-willed and influential peer attempts to win you over to their position by using erroneous information as foundational to their argument. Give a detailed description of how you would respond.
The general manager emails you a consultant’s report which outlines findings about a recent decline in customer loyalty. In the email, the GM also makes a general statement that “the path forward seems obvious,” that she wants you to “take action now,” and that she sees “no need for any discussion.” After reading the report, you determine that there is more than one way to proceed. In detail, describe what you would do.
Situational Drill Down/Follow-up Questions
What assumptions are you making?
Why did you choose to proceed that way?
How did you determine that was the best course of action?
What other unknowns have you not accounted for?
Describe all the elements of your decision making process.
List three other pieces of information that it would helpful to know about in advance.
Implementation Thoughts
- Make sure you consider the potential liability of testing.
- Use only the highest quality tests and expert interpreters. Tests should have reliability and validity coefficients in the high seventies or eighties if possible.
- Remember — all tests and interview questions have built in error.
- Certainly, hiring decisions should never be based solely on test results.
- Optimally, integrate both testing and targeted, personalized questions into the process.
- Consider assigning a 1-10 rating for answers to follow-up questions if the same questions are asked of all candidates.
- Ask references to rate the candidate from 1-10 on: accurately understanding problems, analyzing evidence, and making good decisions. Then ask them to back up their ratings.
- Consider additional measurements of resiliency, impulse control, and other stress-related areas. High performers need to have a high tolerance for stress in order to problem solve when it counts.
- Remember that formal education and degrees do not necessarily translate into critical-thinking ability. Aristotle once said “It is the mark of an educated mind to be able to entertain a thought without accepting it.”
- Take the test yourself.
Photo from Bigstock