AI is the buzzword of moment, and perhaps the buzzword of the decade. Research firm Gartner considers AI among the top technology trends for 2019. Billions of dollars are being poured into investments for AI, from governments and private companies. And the hype is everywhere, from promises about driverless cars and neural implants to claims that AI will provide a solution for every problem.
And there are the predictions about the impending apocalypse where most jobs will disappear and the world will be ruled by an all-controlling AI. But despite huge investments in the technology, the applications we’re seeing today are largely automating narrowly-defined tasks that mainly require pattern recognition. This is why products for facial recognition have become so widespread, but hardly anything exists that can help with creative problem-solving. Driverless cars are a long, long, way off from being commonplace or just available at all. We are entering what Gartner calls “The disillusionment trough” of the hype cycle.
Getting to the Next Level in Recruiting
Visit the expo hall at any recruiting or HR-related conference and just about every other vendor is claiming that their products include AI. The fact that there’s no such thing as AI matters little. What’s called AI is machine learning — the use of computer algorithms that improve automatically through experience. Machine learning is a way by which we may, one day, get to AI — the flexible, general-purpose intelligence of the type which allows an individual to learn to complete a vast range of tasks. But we’re not there yet. Just about every recruiting product that claims to use AI — chatbots, sourcing tools, resume evaluation products, etc. relies on machine learning, not AI, to function. Many of these products do add significant value to the recruiting process by automating some highly labor-intensive and mundane tasks that few recruiters aspire to do themselves. Chatbots are a great example — they can simultaneously contact hundred of candidates, screen them, and schedule interviews, for a fraction of the cost of having a human do the same.
But recruiting is more than a collection of mundane tasks that if individually automated by machine learning products would mean that human recruiters were no longer needed. Getting to the next level of automation will require solving some fundamental problems necessary for machine learning to work. While it can help make recruiters more productive, it would be more helpful if it was more predictive. For example, specifying which candidates are most likely to leave their current employers; among qualified candidates, which ones are most likely to appeal to a particular hiring manager and be chosen for an interview, and ultimately succeed. Current predictions about which candidates are more likely to leave tend to be weak and lacking in evidence as to accuracy.
An article by Peter Cappelli and Prasanna Tambe at the University of Pennsylvania explains why these problems are not easily addressed by machine learning.
- Lack of data. Data sets in recruiting are small by the standards of data science, making it difficult to train algorithms. Getting the data needed to make a prediction about a candidate’s likelihood of leaving can require having access to multiple sources of information other than the resume, such as social media posts, which are not readily accessible or easily interpreted.
- Anchored in the past. Training an algorithm to select candidates most likely to succeed at an employer will likely use performance data from existing employees. This can perpetuate discrimination, as Amazon found in a recent attempt to develop an AI recruiting product. (An upcoming conference session will cover AI legal issues for recruiters.)
- Success in most professional jobs today requires being able to work in a team. That is, success is dependent on others. It’s difficult to develop the data needed to gauge the effect of a candidate on a team’s performance.
- Lack of flexibility. A key limitation of recruiting products that help select candidates is that they can limit the type of candidate who gets hired to a very narrow set of criteria. Once we rule out hiring candidates who are not chosen by the algorithm, the opportunity to see whether other attributes might lead to better performance diminishes and may end; say, if job requirements change or if new attributes appear among candidates. The opportunity for the machine-learning algorithm to keep learning disappears.
Addressing the above challenges is not easy. Some of the problems, such as identifying what makes an employee a “high performer” are structural. Outside of jobs such as sales, there’s little agreement on how to evaluate success. Performance-appraisal scores are notoriously unreliable and lack much validity. Absent a usable data set, it’s essentially impossible to train an algorithm to find high performers.
What’s Next for AI
Major changes in AI in the near term are likely to be in natural language processing. Microsoft’s NLP group has developed technology that makes the voices of computers nearly indistinguishable from recordings of real people. This can make chatbots more engaging and natural, and can be used for services like converting digital text to audio in ways that are more pleasing to listen to. Microsoft also continues to make improvements to speech translation products, which have the potential to make it easier for recruiters to work with candidates who speak other languages.
A major focus of AI developers will also be addressing the need for transparency or getting past the black-box problem. That is, showing how predictions are made by AI. The need to do so comes from requirements like the type imposed by the EU in the GDPR (General Data Protection Regulation), but also to promote trust in the technology. This is particularly relevant for recruiting products. Employers that rely on AI products to make hiring decisions risk liabilities in the future if current technology is later determined to be discriminatory.
Given other developments in AI that are considered breakthroughs, it’s difficult to see that any will have much of an effect on recruiting. Improvements in recruiting products will likely be incremental at best in the coming years. What’s needed is machine-learning technology that allows for better predictions using less data or unstructured data. AI is here to stay, but as technologies evolve, the “wow” factor reappears less frequently and often requires developments in other fields, so temper your expectations appropriately.