But, tools that are not related to the job and are not used in the right manner can lead to severe limits. Its not just the test, but the mindset of the user and the value they place on the correct implementation and evaluation.

Also, im the first to admit that employment tests are not perfect, not even close. But even with imperfection, they still have huge ROI.

@Dr. Handler: I agree- a blended approach should work best. About the “Singularity” per Ray Kurzweil: I think it won’t be 100 years away, I thin it will be 30 years away, and ALWAYS WILL BE… In a happier vein (as Edward would say), heard a good one:
Having techies design social networks is like having a Mormon bartend….(No offense ment to techies, social networkers, members of the LDS, or bartenders).

Happy Friday, ‘Cruitaz!

My view is that AI helps humans do their jobs better. In the case of matching people to jobs, its hard to argue that the job board model is a good one. So, what is wrong with software that helps “direct traffic” and get applicants to the right place for a two way dialogue? This is not the same even as a more down the funnel assessment. It is a complimentary piece of the process.

Within these tools themselves, again its a blended approach. Use AI to help but include real measurements that have been proven to measure the traits they purport to reliably and accurately. Its hard to argue that if a trait is required to perform a job, that some indicator of that trait in the hiring process is not worthwhile. It is NOT the case that these indicator should do all the work. Effectiveness requires expert human judgment. AI and technology support this and add efficiency to the process.

My two cents

As a junior programmer, I was involved in my first work in implementing AI type software back in the 1980’s. A group that I was working with for a large railway built a train dispatch simulator (software), and then tested it’s capabilities against actual experienced train dispatchers. Here is what they found.

1. The program outperformed the average train dispatcher, especially under conditions in which there was adequate, non-ambiguous data.

2. The program did not outperform the best dispatchers, especially when they were fresh and not tired, and provided there was adequate, non-ambiguous data.

3. The program did not outperform either average or the best dispatchers when data conditions were ambiguous and contradictory.

Although I have maintained an interest in this topic ever since then, and have been involved in several similar operational projects to build similar smart software systems, I believe that the following is still true. (I admit however that I am not up to date with the latest findings in the academic research literature on this topic.)

1. Software can outperform human beings in situations where the problem is well structured (e.g. chess), it is possible to search through an extremely large file of established cases to find similar situations (millions plus) in a very short period of time (seconds or parts of seconds). The data defining the specific problem situation is well structured and unambiguous. I believe that the targeting software used in weapons applications, and some of “smart diagnosis” software used in things like equipment maintenance and repair software tools are of this nature.

2. Software can outperform human beings in situations where pattern recognition is the core of the problem space, and heuristic (fuzzy logic) algorithms allow the rapid searching of many “on-file” patterns until a best probability solution is found. I believe the advances in speech recognition software and search algorithms have some of these characteristics.

But I don’t see “recruitment”, which involves understanding the particular performance requirement as expressed in words by a particular manager at a point in time in an organization as being this kind of problem. Therefore, I believe that we have a way to go towards having a software solution that will do this.

Like Lou Alder, I have found that a disciplined approach that focuses on performance (both what is required and what the candidate has done and can currently do) is the best way to hire top talent. It has always worked for me in the organizations that I have managed. Although I have always been frustrated by the need to continually remind my hiring managers and recruiters that we need to stick to the disciplined performance based recruiting process, and not just go with our “intuitive” likings for people. Reading “Honest Signals” by Alex Pentland of MIT showed me that I will probably always have this frustration as long as I am involved in situations in which people interact with people. Our instinctive abilities have a tendency to take over, especially when we find the disciplined process that we have to follow takes more ability that we have or more energy than we we want to put into it.

Happy Friday to you,

Roelf

Happy Friday,

Keith

Assessment companies should explain how they do things because they claim to have a level of technical expertise. If they were individual members of the American Psychological Association, they would be professionally bound to do so. Because they are Internet companies they are not? Sorry, don’t accept that.

I am an Internet enthusiast. I have been actively involved in developing commercial programs and packages that run on it since the mid 1990’s. At one time, I ran an e-commerce software development organization for a large financial institution. The technology is not the issue. The underlying integrity of the vendors is. It would be if their products where on sold on the Internet, but printed on paper. It still is, when their products are sold on the Internet.

Lou Adler is on to the issue here. Recruiting is hard work. The Internet has made some of the administrative processes easier, expanded our ability to reach out into the marketplace, and given us the ability to make a lot more things available to a lot more people. But it cannot and will not replace experienced judgment. No matter what you think, a piece of software, even a piece of artificial intelligence software, is still a construct that is built up out of a series of inter-related “if then”, not the ability of a person to make complex judgments that are part logic and part pattern recognition and part experience based intuition. Minds (using your head to paraphrase on of the commenter’s here), will, for the foreseeable future, remain more complex judgment making gear than software.

Human beings do more than make judgments. They also engage in fads, look for silver bullets, seeking ways to do things they have difficulty doing. After a couple of days of visiting most of the websites in the original article, I have to come to the following conclusions.

1. The sudden growth of these firms says more about their evaluation of the state of the “recruiting marketplace” than they do about the real progress in the state of assessment. These firms believe that there are lots of recruiters out there, both in-house and third party, who will buy these promises of making things easier. They also believe, explicitly or implicitly, that these “buyers” are not very deeply educated about psychometric theory, and not very experienced with the effective use of psychometric tools in recruiting. If this conclusion is correct, then I think it is a sad statement on the professional state of the recruiting profession.

2. Lou Adler and others are right. Talent is here to find, assess, and persuade to move, especially if it is already well employed. In the face of such difficulty, people look for “silver bullets”.
3. An often lot of recruiting work gets by with placing “just good enough” candidates. But after the lessons of books like “In Search of Excellence”, why are we surprised that many organizations are content with “good enough”, especially if they are making money or surviving nicely.

Finally, a prediction. Four to five years from now, most of these promising new firms will not be around. The market is in the long run a harsh weeder out of those who claim and then cannot delivery on their promises. For me, a person who has a background in both the Internet and in work place psychology, that prediction might be the most unsettling result of my initial enthusiasm of several days ago.

On the dollar value of good assessment, Mihai mentioned that nobody seeks “brilliant” electricians, but that’s quite untrue-ask any foreman and she will tell you that the best electricians are WAY better than their peers- it shows up in their work, their interactions with team members, their lower rates of re-work, etc.

Lou, people WILL take a winding path to a position if they think there is a worthwhile reward at the other end …in fact, lots of high value sales situations use baby steps of commitment to move people down the track. High value employers like Google can make people jump thru hoops to no end with smiles on their faces…. If you are a lower value employer, the trick may be the sales job, not the strategy per se….

People are dumb about how computers are smart. Using a computer to make decisions based on emergent properties is the hard way- and you get housing crashes and bad draft picks.

Using a computer to chunk all of the probabilistic outcomes into an array for trained and experienced humans to make decisions is the smarter way, but markets and buyers will never be dissuaded from the dream of pushing a button to get a result.

While it is great to see so many tools, they are all fundamentally addressing a similar thing – parsing down the applicants to get to more of the candidates – at least the ones found online.

And therein lies the real problem.

The scarcity of talent is the real problem and this exists because 70% of potential candidates do not put their professional information online and make it accessible. Therefore, you still have a lot of manual networking and discovery going on 18 years after online recruiting surfaced.

In my opinion, these tools may help parse some of the “noise” today, but they are not going to expand the candidate pool much beyond what it is today. And that is what recruiters really need to increase their effectiveness and job placement.

How many fully-employed top people do you know who are willing to apply and endure these assessments for other than entry-level jobs? A scarcity of talent strategy designed to attract the best, not weed out the weak, is a better option.

While these assessments are useful confirming indicators of success they are not great predictors of it. They also eliminate cultural diversity, eliminate high potential candidates who are light on the skills, eliminate high potential candidates who have a different mix of style, personality and experience, and eliminate every person who is not willing to endure the assessment process, which is about 80% of the fully-employed workforce over 25 years old. Why would anyone contend missing this group is a good thing. It perpetuates the idea that the 20% of people who are looking will always be looking, and everyone else will get promoted or assigned better jobs through internal moves.

Let’s have a real honest discussion about the value assessments at the strategy level. If the future is about better assessments, in my mind, it’s a very dark future.

Stephen

1. identify some true “high performers” who are currently doing the job.

2. profile them in a way that actually identifies some thing about that that they all share – whether it is a personality trait or a competency – whatever

3. make sure that what you have identified has two characteristics – a) it is something that is truly unique about them not something that they happen to share with other people and b) understand / explain in an understandable way the causal connection between their unique thing (or things) and their high performance on the job

4. If all of that holds together, find or design or build a tool (a test, a simulation, a role play etc) which reliably identifies presence of this “thing” in candidates

Do all of this and you have a reasonable chance (it is a probability game after all) of predicting that this person MAY perform well on the job – you still have to worry about things like motivation etc but ….

So why do most of the companies in this great article

a) not talk about how they do this
b) promise they can predict on the job performance based on their “tests”

and so on

It all feels a bit like an well-intentioned con game to me

1. We know recruiting is not easy.
2. We know most hiring managers really don’t have anything other than a “guess insight” into what differentiates the strong performers in their direct reports from the average or weaker performers
3. We know that few performance appraisal systems are based on metrics rather than other more subjective factors

so consistently recruiting high performers remains a really hard thing to do
but everyone what to believe they do it
so that makes them vulnerable to the well intentioned con

we can help you do this (the promise that people what to hear in the con) and you know what we make it easy (appealing to the “greed” factor that is needed for a con to work)

I really hope that I am wrong.
Because I want to be excited as I was in the beginning.

In the meantime, I will continue work with hiring managers building performance profiles against which to recruit, even though that is often hard, frustrating and even unappreciated work.

Roelf

Interesting to see you acknowledge that “determining the effectiveness of pre-hire assessments has been difficult”.
That’s a pretty important thing to say. But does it really follow that the main cause is “challenges with collecting post-hire performance data”?

I think that could be part of it, but not the driver.

For 15 years I have been sawing away at the notion that the problem with pre-hire assessment is that it is too focused on individuals, not groups. It seems that high impact work is nearly always done in team settings, making team performance the driver of outcomes more than individual performance.

When a result is greater than the sum of its parts, the term used is “emergence”, and emergent phenomena are very, very hard to predict (although they can be reasonably bounded).

When you assess an individual, and then expose them to your organization for awhile, the person you assessed no longer exists. The breakthrough will be in learning to predict what THAT person will be like, and how they will affect the team members you have exposed to them.

In sports they call it “chemistry”, and Vegas bettors (and battlefield generals) know just how critical that can be….