Reference Checking Approaches: Is it Time to Blow Yours up?

Reference checking of candidates is conducted by nearly every firm in the Untied States. Some firms conduct them internally, while others outsource the process. Unfortunately, both approaches are often executed with little concern for accuracy or effectiveness.

In fact, it is not uncommon for reference checks to be completed by recruiting coordinators with little subject-matter expertise, using a generic set of questions that offer little in the form of insight into a candidate’s actual past job-related performance.

The bulk of today’s approaches and processes exist for no other reason than to offer an out to the human resources function in case of a critical incident involving an employee.

In a fast-changing world, quality reference-checking approaches are becoming more important due to:

Business demand for assessment processes that help ensure candidates are not being screened out due to personality conflicts or out-of-the-box thinking. In modern enterprises, innovation is job one and that requires diversity of thought, not homogeneity. Reference-checking systems can offer insight into a candidate’s ability to complete the work regardless of organizational fit.
The growing dispersion of the labor force requires that organizations be able to adequately assess candidates who may not be as readily available for traditional face-to-face assessment approaches.
New legislation regarding privacy around the world requires processes that are more tightly controlled, documented, and geographically centric.
As the world’s labor population becomes more visible to employers and the demand for scarce labor increases, managing processes to create or build a relationship with possible candidates in the form of references will be critical. For example, how you conduct a reference offers insight into how your organizations values talent. What message does your current approach send?
Reference-checking processes can offer insight into a candidate’s normal behaviors, as opposed to those that might emerge under the stress of other assessment approaches.

While reference-checking approaches will never be as valid as say, content valid simulations when it comes to predicting employee productivity, it is important that organizations attempt to make their processes as valid as possible.

Unfortunately, I can count on one hand the number of organizations that have implemented a process for validating their reference-checking approach. I realize that doing so would expose the organization to liability if they opted not to correct a process that could not be validated, but would your organization honestly accept the notion of not testing the quality of their product?

This is a bad situation, and one that has gotten even worse as HR functions have become laser-focused on pursing incremental efficiency in transactions without regard for effectiveness. It would not surprise me to find that the accuracy of reference checks in predicting performance is somewhere in the single digits for most enterprises.

If you want to stop assuming that yours works, review this list of common problems with a majority of corporate-run processes.

Inaccurate Results

Here are some common reasons why reference-checking results may be misleading:

Applicants select references who they feel will give positive feedback.
Applicants often “coach” their references to say the right things.
Innovators and out-of-the-box thinkers are often screened out because their references provide stories and examples of behaviors that do not fit the norm or desired profile.
The reduced tenure of talent in many modern organizations can make it difficult for candidates to stay connected with those individuals who were in a position to truly observe their performance.
Often, under-trained employees conducting reference checks “prompt” references for positive comments through the language they use.
References may have hidden agendas that cloud their objectivity. For example, professors may want to help a student get a job because schools are ranked based on their ability to place students, or a current manager could lie in an effort to offload a weak employee or to keep a good one longer.
Unless you take pictures or fingerprints, you have no way of knowing whether the applicant is the same person you are investigating. There are numerous documented cases where people fraudulently use someone else’s “name” to get a job.
Even simplified references like “dates of employment” can often be inaccurate because of bad recordkeeping or the fact that the individual shifted jobs many times within the corporation. Discrepancies and titles can also lead to candidate rejection even though the title mix-up is due to recordkeeping errors.
Reference checkers seldom ask for actual performance data or performance appraisal summaries, instead relying on personality and behavior traits. Granted, many firms would be reluctant to give actual performance data to a reference checker.

Metrics and Measurement

Here are some common measurement errors in reference checking:

They are not given a quantified “score” so that comparisons are easy and a clear delineation between a positive or negative reference is clearly defined.
There is no feedback system from performance management to see whether new hires who had performance issues also had lower reference scores.

Process Errors

Here are some common process or administrative errors in reference checking:

No one “owns” the overall process and is personally accountable for high-quality six sigma type execution.
In many organizations, a junior person is assigned the reference-checking task.
Hiring managers and recruiters are often not trained in the reference-checking process.
The same process and scoring is used for all jobs classes (auditor references may be more important than janitor references) and reference questions are not changed based on the skill and experience requirements of each unique job category.
They are not recorded, so you have no record of what has actually been asked or what has been said, so the person could deny it later if legal action occurs after employment is denied due to references (references, like any selection tool, must be valid and reliable).
The reference volunteers personal or illegal information and there is no evidence that it was not used in the hiring decision.
“Personal” references are requested and then checked (which are generally not job-related and may be illegal). Personal references are self-selected, so they generally turn up no negative information.
The expense, language, and time zone differences may mean that out-of-state or international references are not checked at all.
Reference checkers have no way of knowing whether the individual they are talking with actually has any direct knowledge of the individual’s performance. Errors occur when the individual supervisor is unavailable and someone else volunteers to “fill in” and do the reference from memory (in fact, almost all references that are done outside of HR are done from memory, without referring to the actual employee file).
Reference questionnaires seldom individually weight the questions to make sure that the factors with the highest correlation get the highest weight in the reference assessment.
Some references are done “ad hoc,” without standardized questions.
Individuals in “protected” groups may statistically have higher “weak reference” scores, which could result in some adverse impacts and legal issues (especially credit checks and criminal record checks).
There is a variation in the accuracy of reference checking (different answers are often obtained when HR does reference checks than when managers do them.
Some references are checked without the signed permission of the applicant (the permission request and signature is always present on a paper application form but it does not accompany unsolicited resumes or even most online resumes submission systems).
Most reference checking systems are paper-based, and there is no electronic record available to other recruiters and hiring managers that allows them access to past results.
References are often considered “pass fail” and are not weighted with other parts of the selection criteria.
References are done just once, at the time of hire. However, they are never updated during employment, when it is possible an employee could have committed some criminal offense outside of work.
Over 90% of all references are positive, so why bother?
There is no evidence that the “would you rehire them?” question has any accuracy.
The checking of references is often done too early in the screening process (for all applicants) rather than just for the finalists, thus wasting a lot of money.
“Old” references (past three years) may be given the same weight in the hiring decision as current jobs, which would be an error.
Hiring people contingent on successful reference checking might seem like a good idea, but even when issues are raised later on, the individual is seldom discharged. Often, managers don’t want to go through the hiring process again.
Doing reference checks via e-mail can have advantages, but it’s harder to establish the identity of the person you are contacting.
Many reference-checking systems rely on the phone numbers given to them by the applicant. This can result in serious fraud (always obtain the phone numbers yourself).
Outgoing reference checks are seldom assessed or monitored either, so they too can have negative consequences and high liabilities.
Reference-check systems seldom determine the appropriate “right and wrong” answers in advance, so the interpretation as to whether the information results in a positive reference is subjective.
Most exit-interview processes fail to get the exiting employee’s permission to give a future reference. In addition, managers seldom put together reference information at the time of termination, when it is most likely to be accurate.
Many reference-check systems immediately reject a candidate once negative information is found, without contacting the candidate and giving them a chance to justify the discrepancy.
Because responding to references isn’t a high priority, phone calls from references may take a long time before they are returned.

Four Must-Do Steps

Here are some critical things you must do to have a world-class reference-checking system:

Use “mystery shoppers” to act as references in order to determine whether the person checking the references does it precisely as the process requires. If you hire an external vendor, also check their work periodically.
Regularly measure the correlation between reference checking scores and on-the-job performance appraisal scores.
Focus your efforts on jobs where public safety and theft are real possibilities.
Look at external reference-checking vendors because there are specialists in this narrow area.

Conclusion

It’s a fact that most reference-checking systems are weak at best. They emphasize the positive, but they seldom result in accurate or reliable reference information. Regardless, most corporations still do them, mostly because of the fear of repercussions when an employee causes problems. Should you opt to do them, develop an approach that minimizes the errors and maximizes the accuracy of information obtained.

I bring this topic up because many corporations that I work with are seeking to hire more innovators and outside-the-box thinkers. Unfortunately, screening out such individuals is the biggest failing of reference-checking approaches. If your application screening, reference checking, and interviewing processes do not have specific design elements to prevent the premature elimination of outside-the-box thinkers, realize that you are dramatically hurting the bottom line of your company.