Receive daily articles & headlines each day in your inbox with your free ERE Daily Subscription.

Not logged in. [log in or register]

Moneyball and Recruiting: The Future of Hiring or Pie in the Sky?

by Feb 20, 2014, 12:29 am ET

Screen Shot 2014-02-19 at 9.31.09 PMMoneyball is getting to be the new buzzword in recruiting. We’re supposedly on the cusp of a data-driven revolution in hiring. And it seems one is sorely needed, judging by the state of hiring practices today.

When NASA was just getting started many of the engineers that were hired were chosen only on the basis of their resume and cover letters. That was the norm for many jobs up until the 1950s. Interviews were not common for jobs where the candidates were located far from the worksite — the cost of travel, and even long-distance calls, made them unaffordable. Then employers started using all types of assessments, which would suggest that hiring must have improved dramatically over the 50 years that have elapsed.

One would be wrong to reach that conclusion.

A recent survey by the Corporate Executive Board found that almost three-quarters of hiring managers reported that they hired candidates primarily because they had personalities similar to their own. Another study of major investment banks, consultancies, and law firms found that many candidates are hired because they share leisure pursuits with the hiring manager! So much for the advice of leaving such items off a resume. Many hiring managers described their hiring practices to be similar to dating. No wonder eHarmony has gotten into the hiring business.

How could we have reached this level of dysfunction? The resume and cover letters that NASA relied on to hire employees produced people who built a rocket to get to the moon, using little more than slide rules. And now we’re hiring people using questions made up on the fly? Blame goes to everything from EEO laws to increased job-switching by employees that has made it less economical for employers to test thoroughly. Hence the desire for Moneyball — the new, new thing in the never-ending quest for silver-bullet solutions in hiring.

Why Moneyball?

As far as baseball is concerned, the concept is simple: use player performance stats to make hiring decisions. And in business we love using sports analogies. We also need new content for HR conferences — social media is getting to be passe. Enter Moneyball — it’s based on a true story, was the subject of a movie starring Brad Pitt, and has a David and Goliath theme. A topic with such roots is good for at least 5-6 years of webinars, presentations, and panel discussions featuring unrepresentative case studies peppered with unverifiable statistics. Makes for great conversations though. For once, the VP of HR and CFO can talk the same language. What’s not to like? Expect recruiting departments to start looking for people like the character played by Jonah Hill in the movie.

But in all seriousness, Moneyball applied to recruiting can make for huge improvements in hiring. The challenge is finding the data needed for this to work. In baseball everything is measured and recorded, and it’s near impossible to fake anything. Not so in hiring for other jobs, where data on candidates is either nonexistent or sparsely available, or getting it is not easy.

Consider an example of how Gild evaluates developers. The company’s algorithms scour the Web for open-source code, and for the coders who wrote it. They evaluate the code for factors like simplicity and documentation and the frequency with which it’s adopted by other programmers. They also look at questions and answers on forums like Stack Overflow for how popular a given coder’s advice is. Gild then scores programmers who haven’t written open-source code by analyzing clues embedded in their online histories.

Achievable Dream or Pie in the Sky

Big data is supposed to solve the problem of Moneyball and talent. Maybe. One can only use data — big or small — if it exists. What companies like Gild do may work for hiring programmers, but what about candidates for HR, marketing, accounting, and other categories of jobs where such data simply does not exist?

But that’s not to say it isn’t possible. Looking at candidates online profiles, posts in professional forums, quality of education using assessments like the CLA (Collegiate Learning Assessment) may allow for at least some hiring practices to become like Moneyball. However, this is not likely to ever be a complete solution; candidates lacking an online history are not incompetent. Also, since it’s possible to have one’s online history scrubbed, it’s also possible to have one created that would make the whole idea of Moneyball worthless.

We’re always looking for simple solutions. Ideally, we’d like a method by which any candidate can be scored like a baseball player. But in the entire history of major league baseball there have only been about 16,000 players and only about 750 active today. These numbers are miniscule compared to other jobs. Tracking performance data on that few is easy. As scoring algorithms become more sophisticated we’ll be able to evaluate candidates better, and the concepts of Moneyball may be applicable to aspects of recruiting — like sourcing, but expecting it to apply to all jobs is likely a pie in the sky.

image from http://www.imdb.com/title/tt1210166/

This article is provided for informational purposes only and is not intended to offer specific legal advice. You should consult your legal counsel regarding any threatened or pending litigation.

  • http://www.matchpointcareers.com Paul Basile

    We have the data now and this should be acknowledged. The main problem is that companies don’t use the data at scale because the means of collecting it are complex, expensive and time consuming. Even with those impediments, the companies using psychometrics dramatically outperform those who don’t. It needs to be acknowledged that that’s how Moneyball works – know what to measure and measure it. Solutions now exist that leverage performance-predicting data without the complexity, expense and time. It’s important to acknowledge that.

  • Richard Araujo

    “The challenge is finding the data needed for this to work.”

    Bingo.

    And while I’d like to see examples of what Paul is referencing above, it’s likely that such data will never exist or be accessible. There is a strong incentive to not collect such data, and keep it confidential if it is collected. And in my experience on the corporate/generalist side, it’s in no small part due to fear of lawsuits and just general exposure of incompetence. I’ve seen no shortage of people identified as “poor performers” who completed all their work, on time, and to their manager’s standard, and their ‘poor’ performance was due to some juvenile personality conflict. And how many “poor performers” are labeled as such by managers who are not up to their own jobs?

    I think in this case the data that many people think would be predictive is actually a lot less relevant, and the managers who are making decisions based on common interests and personality may have a point. Because, if the key to doing well at any given company is more of a human relations issue than job skills, which are actually easy to find and assess, then why should those managers not make their decisions based on such data?

  • http://www.thehireauthorityexecutivesearch.com Edward Woycenko

    In my opinion, the majority of hiring problems begin with the hiring managers who do not have definitive goals and objectives by which to begin the search process. IF you look at performance based hiring, or topgrading, the consistency of top performing people using verifiable data solves the performance related problems from an individual perspective. The problem tends to be that most managers have been promoted from the ranks without having been mentored or mandated to obtain skills that would serve them well in a management function. Promotions statically fail 70% of the time. Fear of job loss and having your weaknesses exposed can be masked by hiring people like themselves or by hiring people who have similar interests to the hiring manager. By doing so, you can avoid having a serious conversation about business by discussing shared hobbies and interests. It takes work and investment by corporations to enact a methodology which generates recruiting and hiring success. Waiting for “A” players to show up at your company through social media, job board posting, or advertising on a company website is not going to happen. If companies were truly motivated to hire quality people and build a worldclass organization, they would adopt performance based hiring and topgrading techniques which would accomplish those goals. Unfortunately, companies and managers seem quite content to hit the easy button, which at best, results in mediocrity.

  • Bill Craib

    An interesting read and I agree with many of your points, thanks Raghav. On the other hand, I think assuming that the Moneyball analogy to recruiting is only about data misses a good part of the point.

    The beauty of the Moneyball story to me was not so much how the A’s rethought what talent looks like (using data to do so), but that they decided to do so in the first place. This is a team that looked hard at it’s business strategy, said our goal is to win, and figured out what that needed to change to get there. Data was the vehicle they used, but it needn’t be the only one.

    As Talent Acquisition Strategists think about how to make their organizations more capable of meeting their business objectives there are a lot of opportunities to do something similar. Some of those certainly could be enabling better hires through the use of data, but greater use of contract talent, increased visibility into the internal talent supply and becoming the output process to strategic workforce planning initiatives would all have as much or more impact.

    Thanks again for advancing the conversation.

  • http://PeopleAssessments.com Dr. Tom Janz

    Great to see another insightful article from Raghav. When you describe the detailed data available on baseball (and most other individual and team players), you reference the performance (or criterion) data side of the hiring equation. Most businesses do a poor job of capturing and reporting performance data at the personal and team level. Sports teams don’t as often measure the psychological constructs that underlie performance (mental abilities and personality factors). They measure the performance, not the person (other than for height, weight, BMI, coordination, etc.).

    Companies attempt at least to measure the person–often using lame personality or culture fit tests that have no evidence of being related to job performance. Still, some companies use well-constructed and validated psychometric measures of performance potential, but then companies do a poor job of capturing and reporting personal and team performance measures.

    Using performance in the minors to predict performance in the big leagues works for sports, but not so much for business. After all, performance in the minor leagues for business is not objectively measured, leaving it up to the candidate to describe their “strengths and weaknesses”. We need business to step up to the plate and improve they way it measures and tracks performance before big data will have a big payoff.

    @Paul We can collect highly relevant predictor data now, and I am sure that for those candidates willing to suffer through the current experience, you (and others) do. But we need to do better on three fronts: [1] Make the experience more engaging (and less like pulling teeth), [2] Make the experience more mutually rewarding for candidates, and [3] Capture personal characteristics (via simulations) not well tapped by multiple click, self report test items.

  • http://www.thehireauthorityexecutivesearch.com Edward Woycenko

    @ Dr. Tom Janz: Performance based hiring works in business. Companies have to willing to buy into a culture of excellence in order for this to happen. Suggest you read “Topgrading” by Brad Smart.

  • Richard Araujo

    “The problem tends to be that most managers have been promoted from the ranks without having been mentored or mandated to obtain skills that would serve them well in a management function. ”

    A VERY good point. Most managers were just – hopefully – high performers, and when an opening came up it was, “Congrats, kid, you’re now in charge of the whole department, good luck.” Management is a skill that needs to be trained and honed, like any other skill. Most reviews these days are full of soft measures too, which I find next to useless. I saw one review that actually included a rating on the person’s ‘sense of urgency,’ a typical bit of corporate jargon that never fails to get me riled up as to how subjective and ultimately useless it is.

    This concept of data driven hiring is sound, however I don’t think the necessary data or honesty about what actually matters will be available for some time, if ever.

  • Shanil Kaderali

    Raghav – great article. Hope you’re well.

    Moneyball is also about acquiring undervalued assets (how a company without the budget of the NY Yankees can compete. In football, Tom Brady or Richard Sherman are examples – ie late round picks) – this is the essence of talent strategy for a david going against a goliath.

    Everyone has to buy-in to the strategy and the data is used to support the approach. It wasn’t overnight that buy-in occurred. We have to sell to hiring managers why this ‘new’ data that is an additional point in hiring process makes this candidate valuable over the shiny/glossy ‘top’ talent. Not an easy sell but doable as long as your boss or influential leader supports approach.

    Now the approach has to be balanced with other talent labor supply mechanisms (internals, contract, outsource, etc)- of which normally, the data on that talent is scarce…

    Agree with many comments specifically that data isn’t always captured easily; how to interpret and manage hiring data is still an issue – Recruiting departments don’t have analytics teams and companies don’t always invest in science (exluding google/FB etc)

    Conversation will be different over time which is good.

    Right now, getting Hiring Leaders to grasp that even minor reductions in turnover will result in revenue retention can be a challenge for many organizations even though it’s common sense (well, to TA and HR folks it is)

  • Keith Halperin

    Thanks, Raghav. Glad you’re continuing to write and to recover…. I think there are two major false assumptions here:
    1) People are rational actors. We’re NOT- we’re riddled with inherent cognitive biases which distort our ability to make objective decisions. In organizational life, these often manifest themselves as what I call the “GAFIS Principles”: Greed, Arrogance, Fear, Ignorance/Incompetence, and Stupidity.
    2) Employees without major incentives to do so (lots of stock, performance-based bonuses, etc.) are motivated toward the interests of the over-all company, as opposed to their OWN interests.

    Consequently, I believe that for the most part, there are too many vested interests in doing things BADLY. If we decided to go to an objective, verifiable, fact-based recruiting decision-making model, too many very high-level people (and those high-level consultants who “advise” them, aka” telling them what they wanted to hear”) would look bad: the “Emperor would have no clothes” and they would see ’the man behind the curtain”. Unless you have some very powerful and secure people committed to objective, verifiable, and fact-based best-practices (letting the chips fall where they may), I fear that most companies will continue to listen to slick hucksters with high-level connections ready to sell the latest recruiting snake oil or “magic bullet” to desperate and not-yet insolvent recruiters and their superiors who fail to recognize that in most cases they are futilely “rearranging the deckchairs on the Titanic” of their companies’ ill-conceived, over-blown, grossly-dysfunctional hiring practices.

    Keep Writing, Keep Recovering,

    Keith

  • Gareth Cooper

    “A topic with such roots is good for at least 5-6 years of webinars, presentations, and panel discussions featuring unrepresentative case studies peppered with unverifiable statistics. Makes for great conversations though.”

    Great Conversations…Well said.

    I have been listening in on many of these conversations and although great in theory, lack simple practicality. It appears that the one guaranteed result I can bank on following these discussions is a sales call from the sponsor of the event.

    The David and Goliath theme mentioned in the article is an interesting choice. There is no shortage of volunteers brave enough for the challenge. The challenge is trying to find a suitable enough set of stones among a myriad of options available to get the job done right.

  • Bill Wright

    I was going to comment. Then I thought long and hard about the issue of metrics and metrics-driven and data-driven and the Saratoga Institute’s HR ROI push back when most folks were still in grade school.

    Then I thought about Dilbert and Catbert the Evil HR Director and his cabal with the Pointy-Haired Boss and the Slick-Domed CEO.

    And I really thought better of making any comment pro or con. The concept is too reminiscent of a couple of episodes of NUMB3RS.

    Bill

  • Darryl Clements

    I wouldn’t consider the concept of Moneyball pie-in-sky, but there’s one key element missing – GOOD, COMMON, OPEN data. Not screaming – just emphasizing that Moneyball baseball stats are 1) hard data that isn’t subjective and; 2) is the same data for ANY player anywhere; 3) it’s available to everyone because it’s pure performance that every club can track on their own – and this is key – or ANY player.

    Most organizations are woefully incapable of actually internally assessing individuals with this level of precision. Suggesting that they can do it externally is probably not realistic.

    The good news about the Moneyball concept is that it can highlight, as Billy Beane and his team did, that scouts (recruiters & hiring managers) should probably examine other critical data. Want to add more leadership in the organization? Start spending time on the granular details of how a candidate actually leads.

    The other unspoken is that company leadership needs to be as open to changes in data-centric recruiting as Oakland was with Beane.

    Most organizations simply aren’t ready or capable of going Moneyball, but they might inch that way if continually exposed and prodded by inarguable data and facts. With that in mind, I’d suggest organizations Moneyball internally where they can control the data and at least have a baseline on the data’s truth and validity.

  • Keith Halperin

    @ Darryl: Well-said.

    “Most organizations simply aren’t ready or capable of going Moneyball, but they might inch that way if continually exposed and prodded by inarguable data and facts.
    If I were a betting man and I had to bet on either Moneyball data or powerful, arrogant, and entrenched interests who’d stand to lose (or at least look foolish) if they went “Moneyball”, you could guess where I’d put my money.

    “With that in mind, I’d suggest organizations Moneyball internally where they can control the data and at least have a baseline on the data’s truth and validity.”
    Control the data = manipulate the data to show what they want it to show.

    -kh

  • http://www.SeanRehder.com Sean Rehder

    Baseball metrics are easy to apply because data is everywhere and available. When it comes to people, meaningful information is extremely hard to get over periods of time of a person’s career. Especially when one changes employers.

    I believe the first step in regards to data having meaning for recruiters in the recruiting process is to get feedback from current employees are potential candidates. To do that, you have to map your employee’s networks ahead of time so that connections can be flagged for feedback early in the hiring process…perhaps even before a recruiter attempts to reach out for the first time.

    This type of data/feedback isn’t the end all, of course, but it is meaningful and available and helps a recruiter qualify their time spent.

  • claudio weinstein

    Great Subject matter. I have been very distracted from my full time recruiting duties over the past year analyzing and reviewing these types of discussions about predictive analytics in employment.

    I am an avid sports fan and when I learned of the Moneyball concept, I was very intrigued about the possibilities of associating that concept with recruiting teams for my clients.

    However, there is one major element that clouds this concept with that of other types of employment…Baseball, like many other major sports, is Unionized.

    e.g. when a pitcher gets a multi-year contract, that contract is honored by the union rules for the duration of its years no matter where the player plays.

    Although I have seen successful employment contracts written at executive levels, the reality is very few employees are able to negotiate such an agreement up front without the power of a union.

    Now I am not promoting for or against unions, I am just making the observation that this article cannot compare Moneyball to recruiting for the masses unless that element is examined.

    To be clear, the intention of my life’s work is to fundamentally increase job satisfaction and job security for employees across the board so that once employed, it is understood that both the manager and employee should maintain a healthy lifestyle both at, and away from their employer.

    Like major league sports players, I believe employees should be allowed get a realistic expectation of staying with an organization/industry long enough that they get an agreed upon amount of time/effort to have a positive impact on the outcome of the objective that the organization hired them for.

    The question is, how do we get employers to agree to these time lines and commitments to employees on a mass scale without Unions?

    My solution lies in technology that exists today and I am avidly seeking partners who share the passion to be on the cutting edge of developing just such a solution that will change employment as we know it.

    Contact me if you are interested to learn more.

    Me@ClaudioWeinstein.com

  • Tane O’Rorke

    I can see a lot of big data/moneyball type applications for identifying people’s interests, values, motivations, likes/dislikes etc and using these to predict performance, engagement, motivation, interpersonal/leadership styles or some other outcome. This type of information is being freely and continuously offered up now via social media, blogs and the like and is become more and more open every day. By the time my niece is my age she will have 20 years of her life, interests, loves and hates documented on the web in one way or another. The richness of this information unfolding, evolving and building over time has never been seen before and our ability to harness this info will only continue to improve. The future will look very different to the past.

  • Darryl Clements

    @Claudio – unions actually don’t make that much of a difference in how Moneyball metrics are and were applied. They’re applied prior to players becoming part of a union while in HS, college, or minor leagues.

    The only thing the unions do at that point is provide for league minimums. In fact, the baseball union seems to have a much longer, better established history of serving the interests of older, more mature players who often fair the worst when it comes to Moneyball applications. An undrafted, non-union prospect happily goes along with Moneyball concepts. In fact, many prospects now use them to track their own value. On the other hand, established (unionized) players probably don’t care too much for the concepts as they often can and do reveal declining value using data and physical/biometric evaluations.

    I have managed in unionized production environments where strong standards to exist on the same level as Moneyball stats. I can tell you firsthand that unions are not even resistant to below-par performance if the standards have been jointly agreed upon.

    Here’s one situation where Moneyball can be applied – which sales and revenue generators are most effective, including following legal, ethical, and policy standards. Much of that data can be concrete if outlined properly. However even that data can be blurred by internal support or resource inconsistencies and imbalances (e.g. new business versus vertical sales aren’t the same and are more a reflection of company belief than salesperson prowess).

    @Tane – It’s just as easy for people to make up data on the web as for it to be real. I don’t think business people will bet their careers or money on much of what’s on the web. If someone creates the next Facebook or Google, perhaps. Something on an imperceptible, personal scale will only be taken in as a sort of. Sadly, there’s more scouring the web for what’s wrong with someone than what’s right. I can almost bet that won’t change any time soon.

  • Tane O’Rorke

    @Darryl – I agree it’s easy for people to make stuff up and there is a lot of poor practice with evaluating this info at the moment. I am talking about the future where we would have 5, 10 even 20 years of data- a real history on a person’s interests, likes and thinking to evaluate – lots of data, big data. I don’t think many people would keep up a façade for that long and besides how would they know what sort of organisation they should be trying to position themselves for 10 years into the future? Its easy enough to assess for social desirability/faking good type attempts now- we will only get better at this. And remember this stuff is peer reviewed. If I started liking Justin Bieber and commenting about him enthusiastically in the hopes of appealing to some group or organisation I can guarantee people who know I can’t stand him would challenge me on it immediately- in public and in private.

  • claudio weinstein

    So nobody else gets confused about my reference to unions, I will clarify.

    Any of the statistics and data that are generated in a filter for predicting the Value of a potential team member, are used prior to their inclusion on that particular team and are done solely by the team organizer for the benefit of the team.

    Since every team requires a league approved contract for that potential player, and the true value of that player only takes place after the contract is signed, the union IS the defining parameter that changes the face of the negotiation compared to a free enterprise system. (Period)

    And to clarify the value of that data that is available, there will always be margins of error in data no matter how accurate it is professed to be…We are talking about humans here.

    You cannot, and never will be able to generate any predictions on humans without margins of error and failure. All you can do is to mitigate the failure by having Trust in your team, Guts in your decisions, and Money to fill in the cracks.

    As far as trusting the web, Everyone is trusting the web now. It is just how much We are betting on it that differentiates us.

    Scour, Scrape, Mine, Dig, call it what you will. This is how things are being done now and will be done forever.

  • Darryl Clements

    @ Claudio – I wasn’t confused at all by your reference to unions. I’ve worked with them and negotiated with them inside corporations for years. Inside companies, unions can and do work with companies. It’s rarely as adversarial as sports is because when employees strike – they don’t have the cushion pro athletes often do. Inside, companies and unions agree on wage scales and employment conditions just like baseball players and teams do. In fact, ball players have come to wage-scaling that looks like corporate union scales in recent years. Used to be that rookies could negotiate whatever they could get. Now, there are rookie wage-scales in some sports and accepted past-practices in others.

    As to the data, I’m just pointing out that “good” at one company may not be “good” at another. It’s even more of a stretch to get “good” from some industries to be even considered “worthwhile” in others. In the corporate world, that’s reality.

    We are now easily able to find out a lot more about people that we couldn’t before, but that information produces a lot more signal than noise because it lacks the situational context that is a prime component of Moneyball stats. It’s not just that Beane sought players with good on base percentages. He sought good players who had good on base percentages late in games, late in the season, and with increases during playoffs or clutch situations and against left-hand closers with high strikeout ratios but who often can be waited out if you can get them to throw more than 20-30 pitches. There will be some situations where more data about someone is available, but it’s not likely to offer that level of detail and context.

    Again, I agree that the concept is good to push towards. I just caution that most data simply won’t hold up to the standard required to make it a good “Moneyball” fact on an individual.

    Data on people as we now know it isn’t nearly as quality as it could be. We’ll likely cross quality thresholds somewhere, but it will probably be when individuals decide to put more revealing info out there. Companies certainly aren’t going to do it.

  • claudio weinstein

    I can tell more about people from HOW they say things than what they say.

    Your paragraphs are long-Tells me you are not sure of what you say.

    You use numbers, symbols and over proper punctuation- tells me you are well educated and well read…Not necessarily a risk taker.

    Use the word fact.- means you are trying to be influential…Maybe too hard.

    That’s just the beginning.

    Now do you think the data is not there?

  • Darryl Clements

    I’m sure of what I say – I’ve experienced it on both sides. Paragraphs are just illustrations – if I could draw them, I would. Can’t. Don’t go overboard with punctuation in every online exchange. Data and facts aren’t the same – data without context or meaning isn’t less than a fact. I don’t care to be influential – just discourse.

    Yes, data is there. But it is without context so it doesn’t reveal much.

    Just as easy to conclude education and reading by knowing who follows and reads ERE posts. Probably don’t have many friends or associates who wouldn’t say I’m a risk-taker.

    Enjoying the article for it’s stimulation – yes. Otherwise, I wouldn’t have crossed paths with you.