Big Data is Hot, But Sometimes, Small Data Is the Better Way to Go

The recruiting and HR world has been awash for the last year with talk about Big Data.

It seems everywhere you turn you read another article describing how all your hiring problems will be solved if you can simply sift through the massive stream of data generated by your recruiting process to find nuggets of truth.

While I don’t disagree that Big Data holds promise (heck, I’ve even spoken on the topic at several HR conferences) I think the rush to embrace Big Data may cause us to overlook the wisdom of SMALL Data.

Just what is “small data?”

What do I mean by Small Data? If Big Data is characterized by massive data sets and the quest for statistical significance, Small Data is exactly the opposite — tiny data sets and the quest for anecdotal evidence.

Consider for a moment your online application process. A Big Data approach would gather all the data on click-through rates and fall-out percentages and candidate demographics — millions of data points — to try to isolate the reasons why candidates are likely to complete or not complete the application process.

The Small Data approach would start by simply going to an unfamiliar computer in a library or at a friend’s house, completing the application process and making observations about it. Was the process easy? Was it engaging? How long did it take? Did it seem redundant?

Or one might simply ask a few candidates “how was the application process? Is there anything we can do to make it easier?” Many times, simple anecdotal evidence of this kind can be stronger than all the data analysis in the world at uncovering immediate fixes to improve candidate experience.

How about this familiar challenge: sourcing hard-to-find talent. A Big Data approach would run candidate demographics reports, analyze the employment value proposition of various competitors to target for talent, analyze supply and demand across various locations, research and investigate niche job boards or candidate databases and so on.

Some truths can’t be found in Big Data

The end result is a lot of data and information, without always a clear picture of what to do next.

While all of these are useful exercises that we often undertake, it’s amazing how much you can learn my simply picking up the phone and calling the last candidate hired in a particular hard-to-fill niche. The conversation is a simple one: “I know you joined us six months ago, how did we find you? Where do you think we can find other people with your niche of experience? What websites do you visit? What magazines do you read? What conferences do you attend every year?”

While the resulting information is based on a single individual, and not necessarily generalizable to the general population, it will almost always contain nuggets of important information which can point us to our next action to find similar talent.

Especially among the younger generation of recruiters, there tends to be a belief that all the answers to the toughest problems can be found on the Internet or through data analysis. Simply reminding them to pick up the phone and make a few calls when they run up against an obstacle can have dramatic results.

The personal, anecdotal information they will uncover will often contain big truths not found in Big Data.