Receive daily articles & headlines each day in your inbox with your free ERE Daily Subscription.

Not logged in. [log in or register]

Improving Interviews by Using Forced-Choice Questions to Replace Yes-No Questions

by
Dr. John Sullivan
Dec 1, 2009, 6:00 am ET

Picture 3Most managers share the fear of hiring a bad fit, no matter how technically qualified the candidate might be. Unfortunately, most attempts to measure fit suffer from a fatal flaw: that being the way cultural screening questions are asked.

Too many managers and administrators-turned-recruiters ask binary questions that make it easy for candidates to guess the most desirable answer. For example, they may describe their group’s team dynamic and ask if it is the type of environment that the candidate finds suitable. Fortunately, there is an alternative type of question format known as “forced choice” that can be used by those in recruiting to garner a much more precise and insightful candidate perspective on cultural issues.

The Problems with Binary Questions in Interviews

Many of the questions asked during interviews can easily be classified as binary or yes/no, true/false questions. While they may be posed in an open-ended fashion, it is clear from the phrasing and tonal inflection what response is desired.

In other cases, the questions probe the existence of behavioral characteristics that any candidate would need to be an idiot to deny.

Examples of obvious “yes” questions:

  • Are you a team player?
  • Do you work well under pressure?
  • Do you share our company’s values?

Unless you are interviewing Homer Simpson, questions like those listed here will result in fairly predictable responses. By asking a question that allows a simple yes or no answer to be provided, you make it way too easy for the candidate to misrepresent themselves and to give the answer that they think you really want.

Consider Forced-Choice Questions

If you wanted to obtain a more precise answer to an important “fit” or preference question, shift to a forced-choice question format. This format requires the candidate to rank a series of possible responses in order of desirability.

Example:

How many hours of overtime would you be willing to work on a routine basis? (Please place a “C” next to those options you are comfortable with, a “T” next to those options you could tolerate, and a “U” next to those option unacceptable.

__ 0 hours per week

__ 3 hours per week

__ 5 hours per week

__ 7 hours per week

__ 9 hours per week

__ 11 hours per week

__ 15+ hours per week

By phrasing the question this way, you require the candidate to provide more precise insight into their real perspectives.

Using Forced-Choice Questions

Forced choice questions can be used throughout the assessment process but are best employed to either knock out candidates or determine a group for inclusion in further assessment activities early on in the assessment process. They can be used in online surveys following the application, during pre-screening activities executed by phone or survey, and during formal interviews. They work great in high-volume hiring environments to limit time spent on interviewing candidates who will not operate well in the environment or culture provided.

Five Sample Forced-Choice Questions

The following sample questions were designed for use in a high-volume retail environment. They cover a variety of job-related parameters.

1. Please rank the following activities in the order in which you desire to focus on them.

(Goal: to identify the activities the candidate prefers to focus on to see if they match the actual mix of job duties.)

__ Working as a cashier

__ Maintaining front-of-store inventory (stocking)

__ Preparing merchandise displays

__ Servicing customer inquiries

__ Maintaining store cleanliness (janitorial)

__ Processing new inventory (warehouse)

__ Loss prevention (customer monitoring)

2. Please rank the following possible work situations in the order in which you desire them.

(Goal: to identify whether the candidate prefers to work in a role where they are isolated and influence decisions solely or as part of a larger consensus.)

____ Assignment to isolated tasks that allow you 100% control over your performance

____ Assignment to tasks that require a small team where others may influence your performance but that would provide more social interaction

____ Assignment to tasks that require cooperation and coordination of numerous other employees influencing your ability to perform but maximizing social interaction

3. Please rate the following skills required based on your level of expertise using a rating scale of 5-Strong Mastery to 1-No Mastery.

(Goal: to identify if the candidates’ strongest technical skills and knowledge areas match the job requirements.)

__ Telephone communications

__ Cash register operations

__ Dealing with customer complaints

__ Managing time

__ Preparing and displaying inventory

__ Managing others

4. Please rate the following management scenarios based on their desirability to you using a rating scale of 1 (highly desirable) to 5 (less desirable).

(Goal: to identify whether the management style that they are the most productive under is similar to the style used by store management.)

__ Manager closely monitors assigned work activities and provides frequent feedback

__ Manager assigns tasks and allows independent work, periodically checking on progress and providing direction/feedback

__ Manager assigns tasks and trusts that they will be executed according to standards, but does not monitor progress or provide feedback until shift is completed

__ Manager does not assign tasks directly, but rather provides a list of things to accomplish, allowing employees on shift to self-delegate and check off completion

__ Manager works alongside employees completed necessary tasks providing instruction and feedback on performance as time permits

5. Please rank the following possible forms of communication managers may use to discuss your performance with you in order of desirability.

(Goal: to identify whether the communication approach desired is used by store management.)

__ I prefer direct face-to-face feedback in a public setting only when needed

__ I prefer direct face-to-face feedback in a private setting only when needed

__ I prefer feedback via written note only when needed

__ I prefer feedback via email only when needed

__ I prefer routine feedback offered during informal face-to-face meetings

__ I prefer routine feedback via formally documented written communications

Advantages of This Format

As you can tell from this very simple retail example, forced-choice questions enable managers and recruiters to quickly screen candidates based on their ability to operate in the environment or culture that will be provided, versus the perfect scenario often described.

Beyond more detailed insight, forced-choice questions provide the following advantages:

  • They force candidates to deliberate more on their perspective.
  • They make comparison of candidates much easier and less subjective.
  • They help aide selection of candidates more apt to thrive in the actual work environment that will be provided.
  • They prevent candidates from dodging tough questions and from offering up easy canned responses.

Final Thoughts

The entire process of interviewing has many weaknesses and inherent flaws that may lead to bad hiring decisions. In fact, one study by Leadership IQ found that the typical assessment process was only 19% successful in identifying candidates who went on to become undeniable performers.

Minimizing the number of binary questions and substituting forced-choice questions is the smart way to improve accuracy. In addition, many forced-choice questions can be implemented during pre-screening activities that could save you lots of time and money assessing candidates who are not an appropriate fit for the job or manager!

If you would like to learn more about pre-interview questionnaires in general, see my previous article entitled A Pre-Interview Questionnaire for Improving Candidate Screening.

This article is provided for informational purposes only and is not intended to offer specific legal advice. You should consult your legal counsel regarding any threatened or pending litigation.

  1. Rob Levin

    I would argue that just as one can easily guess the answer to a binary question, they can also figure out the preferred answers or order of forced-choice questions with only a general understanding of the position and environment.

  2. josie erent

    This article is flawed. Many companies are disqualifying qualified candidates based on the presentation of resume. Let’t face it some people have poor writing skills and cannot explain their experiences on paper. Then there are others who frankly cannot be bothered. That being said you have limited pool of talent to interview including internal candidates that most companies tend to have an strong hiring preference for…yet companies again are limiting the talent pool by refusing in some cases to use the services of a recruiter.

    You can have a perfect scenerio of asking questions…but are you asking the right open ended questions? There have been studies made of the importance of written documentation….which I professionally think is undervalued. When an interview discussion takes place over 60% of what the individuals talk about is forgotten…

    We are dealing with people who work for companies who also force they individuals through non compete clauses to discuss only certain matters. I have had experiences dealing with candidates that have signed contracts that prevent them from discussing compensation let alone other questions pertaining to the jobs…

    In conclusion, we live in work in a complicated business world determined by fear of lawsuits, employer reprisal and human flaws that in some cases are governed by culture, bias, and fear.

    I am not convinced all companies do a great job of hiring…but certainly there are those companies that have a creative approach to getting seek out and know great candidates and focus on the best candidates rather than mediocre average internal performers….

    I say to my recruitment friends everyday…Thank God there are companies that discriminate, underpay, and mistreat their employees. You can certain focus on questions to hire the right candidates but certainly interviews are a two way street. The greatest challenge for most companies is to prevent great employees from leaving which again most companies fail to do.

    Josie Erent
    Talented Minds
    division of Silicon Executive Search
    416-232-0600

  3. David Arnold

    Rank ordering competencies, desires, skills, etc. doesn’t provide information that can be legitimately used to compare individuals–such information is only useful in determining the rank order of such attributes within the specific individual. In reality, one person’s top ranking may be lower than another individual’s lowest ranking. As an analogy, when rankings are provided regarding football teams, it isn’t insightful to compare two teams from different leagues/levels based thereon—-the best NCAA football team is probably not as good as the worst NFL football team.

    David Arnold, Ph.D., J.D.

  4. josie erent

    I simply cannot resist this article which reminds me of a “corporate date”. The corporate princess has her questions and checklist in order to determine her ideal corporate mate(employee). The ideal mate has everything the corporate princess except he uses fowl language and has big nose and lives in the wrong side of town. He earns a great living and the customers love doing business with him. We can continue on discussing questions till we are blue in the face…..but more and more couples get married to wrong people ending in 50% divorce and in some cases with prenupital agreements. The corporate relationships and couple relationships have both become short term transactions that are no longer meaningful. Keep your questions …but than again why should I work for your company when I can work for your competitor for $20,000 more. I have my own questions to ask you but you avoid them. Not convinced. Especially now with more and more information public and on the internet. It is very easy now to screen a corporate employer. Interviews are a two way street especially with talented employees who are being interviewed by several companies at a time. In this market there are too many companies wasting time asking the wrong questions and not hiring the superstars.

  5. Keith Halperin

    I believe this article contributes to making interviewing too complicated:
    1) The successful interview process requires that only two criteria be established-
    a) Can the person do the job we need them to do?
    b) Do we feel comfortable working with person for an extended period of time?
    If the interviewers can not say “yes” to both questions, the person should not be hired.

    2) If Recruiting is doing its job properly (based on my experience), three-five candidates should be interviewed. In the event that there are a number of suitably-qualified, suitably-fitting candidates without a clear front-runner/favorite, there are decision analysis tools allowing for the quick and demonstrable generation of a hiring decision.

    Also, it was not clear to me if the author were
    1) Asserting his opinion (as I did re: the successful interview practice)
    2) Relying upon anecdotal studies, (as I did re: the optimum number of interview candidates, although *….)
    3) Using formal research results.

    I would appreciate if this were made clearer in the future, and will attempt to do so myself.

    Thank you,

    Keith Halperin

    * http://www.teachingexpertise.com/articles/school-business-managers-role-recruiting-and-selecting-staff-4051, http://hrmarketer.blogspot.com/2008/09/picking-people-lessons-from-peter.html,
    http://www.cmu.edu/hr/recruit_staff/Hiring/interviewing.html,
    http://www.mcabarclaykane.co.uk/content/recruitmentProcess.cfm

  6. Jeff Weidner

    Is there a “perfect way” to interview all candidates the same exact way. Of course not.

    I think the Forced Choice technique could give you more insight if the Hiring Manager actually knows what a good fit would look like beyond the skill set screen of can they do they job.

    Certainly better than Y/N Q&A

    A combination of Forced Choice, behavioural questions and open ended questions might be a better route.

  7. John Younger

    Excellent points by all.

    In the end, initial interview questions are asked of candidates every day by coordinators, recruiters and hiring managers, and the answers to those questions have a significant bearing on whether or not the person will be hired. In fact, finding candidates and conducting initial screens/interviews is where a great deal of the recruiter’s time is spent. Inconsistencies are inevitable because humans are asking the questions and hearing the answers through their own filters, knowledge and experience (or lack thereof). How might job specific initial online interview questions be used to help reduce the inconsistencies, lower the risk of a bad hire and increase the probability that the best possible person is hired?

    There appears to be a little confusion about the different ways that the on-line questions themselves can be developed and interpreted. Consider that there may be four different types of multiple choice online interview questions.

    It’s also important to note that the response to one question may not be sufficient reason to disqualify the person, but that an answer might place this person behind others who answered it in a way that better aligned with the job to be filled. In other words, it is the entire question set defines the person’s candidacy, not necessarily just one single question and response.

    1. Questions that have a right or wrong answer.
    To Rob Levin’s point, the possible binary and multiple choice answers may be something that can be guessed or gamed. Another way to look at these fact based questions is that it may demonstrate a person’s basic ability to look something up correctly or find an answer. Let’s say that the current top performers for that job are those individuals who are resourceful and have a knack for finding information so that an informed decision can be made.

    Here’s an example of a question that may provide some insight about an applicant’s talent in this area:

    The population of the United States is closest to:
    - 295 million
    - 33 million
    - One billion
    - 985,000
    - Six billion

    What might an incorrect answer tell us about the applicant? Did this person demonstrate the ability to understand the question, take the action to find the answer and select that option? The obvious answer is that this would have only required that the person open another browser window and type the question into Google. If this person guessed incorrectly instead of looking up something so simple, what might that tell us?

    2. Questions that demonstrate a level of talent to do the job well.
    In the end, the goal is to hire the A+ candidate who make the biggest positive impact in the organization and role. How might the person’s talent be determined with well crafted questions? It will likely not be determined by years of experience because most of us have seen mediocre performers who have 10 years of experience and others with 5 years of experience who are superstars. Consider that a more talented sales person may have received more documented positive recognition. Here is an example of a question that may help separate out the great from the average if previous experience is required. Note that this is also an opportunity for great people to brag.

    Regarding my quarterly sales quotas over the last 8 quarters (may require verification), I have:
    - Exceeded all my sales quotas over the last 8 quarters
    - Exceeded my sales quotas 4 to 7 times over the last 8 quarters
    - Exceeded my sales quotas 1 to 3 times over the last 8 quarters
    - Not exceeded my sales quotas over the last 8 quarters
    - Been in roles that did not have sales quotas

    Let’s say that we are looking to hire a CFO to help prepare the S1 and take the company through an IPO. Here is a question that might help find someone with the talent to do this well. The underlying thought is that if someone were not good at something, they would likely not be asked to do the activity again.

    I have taken the following number of companies through the IPO process (identify investment bankers, prepare the S-1, conduct the road show, manage the public company reporting):
    - I would be new to preparing the S1 and taking the company through the IPO process
    - One Company
    - Two or Three Companies
    - Four or More Companies

    One last example in this category would be for a Marketing Manager whose role will be to develop sales collateral for new online product launches. The hiring manager needs to have someone who has the talent and experience to do this well. Someone who selected the first option (being new) may be an obvious knockout where the person who chooses more than 10 products may be someone that the hiring manager wants to have advanced in the hiring process regardless of the other questions asked.

    I have directed the preparation of all collateral and sales tools for successful product launches and communication engagements for the following number of online products:
    - I would be new to developing sales collateral and marketing materials for new online product launches
    - 1 to 3 products
    - 4 to 6 products
    - 7 to 10 products
    - Over 10 products

    3. Questions that convey a preference.
    The benefit of a question in this category is find a good alignment between the hiring manager’s style/beliefs with those of the applicant. There is no right or wrong answer.

    Under the stress of a tight project deadline, if it appears I will need more time to complete the project I typically:
    - Delegate more and rely on team support
    - Renegotiate the deadline
    - Work 24 hours per day
    - Do the best that I can and not worry about it
    - Throw myself down a flight of stairs to avoid the issue (this option also conveys the hiring manager’s sense of humor)

    My preferred management style is:
    - Clear top-down delegation and decision
    - Closed book with decision after consulting with management
    - Closed book with decision after consulting with staff and management
    - Open book with decision after consulting with staff and management
    - Open book with decision by consensus

    4. Questions that help communicate the personality of the organization.
    Keith Halperin’s post highlighted this: Do we feel comfortable working with person for an extended period of time? Rather than simply think of questions as a “hurdle” for applicants to traverse, well developed questions can actually engage the person and help communicate the personality of the group and hiring manager in a compelling way. Much has been written about the importance of the personality/cultural fit between the new hires and the group with which a person will be working, so what are some of the quirky things that the group’s top performers value? This is an opportunity for a hiring manager and group to convey that personality is a way that gets the right people more interested in the opportunity.

    My favorite computer game is:
    - Halo
    - World of Warcraft
    - The Sims
    - Diner Dash
    - How can I pick just one, computer games are my life!
    - I loathe computer games

    My sense of humor is closest to that of:
    - Monte Python
    - Jim Carey
    - Eddie Izzard
    - Rush Limbaugh
    - Rosanne
    - Not applicable – humor is highly overrated

    Given the significant variations and inconsistencies of how initial interview questions are asked verbally and interpreted by humans, we have seen significant evidence that the theme of John Sullivan’s post is right on point. That said, meaningful questions can be quite difficult to write and even more difficult to have the right scoring and weighting structure behind them. While most Applicant Tracking Systems have a structure for asking questions, they are not designed to address these two issues.

    The initial online multiple choice and open-ended questions can be an opportunity to find the rock stars. They will likely never replace the in-person interviews and don’t need to. But, used properly in the right framework, they can lead to great hires in record time even with large volumes of applicants.

    As a gift of a smile and to highlight how these four types of questions can be used together for one job, here is one question set that you might enjoy. It’s for the Notre Dame Head Football Coach:

    http://www.accolo.com/press_detail/notre_dame_head_football_coach_search_gets_professional_help/

Post a comment

Please log in to post a comment.

Note: You need to sign up for an account on our new commenting system if you haven't already done so — even if you have an existing ERE account. Find out why »

Login Information