How to Be Sure You Can Trust Your Employee Survey Results

What if I told you that 26 percent of your employees either blatantly lie or inadvertently misidentify demographic questions on employee surveys?

If you’re like most managers, you’ll immediately distrust the survey process and its reported data — that’s completely fair.

The fact is, when employees are asked demographic questions (e.g., department, manager, tenure, job class, age, or gender), 26 percent of them respond incorrectly. Perhaps these employees lie for fear of excessive respondent transparency; perhaps they lie because they distrust the intentions behind the survey and seek to sabotage its validity; or, perhaps they misidentify because they simply don’t know the answers.

Consistent errors in survey responses

Whatever the reason for the inaccuracies, the statistic still holds true; we’ve tested and validated this conclusion with a broad set of our clients, and continue to find the same error rate.

How do we know? When we conduct employee surveys we can track responses based on email address or some other unique identifier.

We receive a file from the client that includes demographic information like tenure, department, manager, etc. and match it to the unique identifier. On some surveys we asked for demographic information on the survey and then compared responses to the data provided from the employee file to find the discrepancies.

While cumulative survey responses are accurate in aggregate (the questions that deal with engagement and satisfaction), these responses cannot be effectively separated by department or manager by relying on employee responses to demographic questions. This is especially important when you are trying to report down to front-line managers and conduct organization-wide action planning.

Making surveys more accurate

Not many things are more embarrassing than providing the operations manager a report for her department that lists 122 total respondents, while her department only has 97 employees.

Whoops!

The solution? Design a survey that codes demographic data behind the scenes. Instead of asking employees identifying questions (which can cause people to not participate), use a system that allows you to track responses yourself. This is the practice we employ, and the strategy we recommend to all of our clients.

By making demographic data a preprogrammed part of the survey, your results will be just as accurate as the company’s HR records.

Now I know what you are thinking: “What about confidentiality? What if employees find out?” Transparency is the key. If there is a low-trust environment, then wait a year or two to track demographic information.

The multi-year filtering approach

During the first year, report the data at high levels and share the results with employees. During consecutive years, filter the data down the management chain so employees can meet to discuss what to do in their work groups.

Only the executive team and HR should be able to access results covering all demographic categories and departments as long as data is provided in groups of five responses or more. This will protect anonymity and confidentiality.

When you’re ready to start this year’s employee survey, ask yourself how you want to track responses. If you plan on using demographic data gathered from the survey, don’t plan on being able to deliver accurate reports to department managers.

This was originally published on the DecisionWise blog.