Cheers,

Keith “Keep it Short and Simple” Halperin

Also, note that predicted on the-job-performance correlates directly with the variance among the raters for each of the 10 predictors. As a result of this we suggest that when the interviewing team disagrees by more than plus or minus .5 points on the 1-5 scale no yes/no decision be made. Additional assessment data is then required to justify a yes/no decision.

When there are more than 3-4 interviewers the process goes out of control and nobody ever agrees on anything unless there’s a strong facilitator in the room. Getting the hiring team to even use the rating form is pretty difficult too, but the results are clear that when used as described predicted accuracy increases dramatically.

This wasn’t exactly my question- I meant to ask (all else being equal), what is the range of optimal numbers of interviewers and interviews to get both good and practical results?

In the meantime, I found an interesting paper
THE CONTRAST EFFECT IN A COMPETENCY BASED SITUATIONAL INTERVIEW
http://docs.google.com/viewer?a=v&q=cache:5G2mw2XgK54J:https://dspace.lib.cranfield.ac.uk/bitstream/1826/3958/1/Contrast_effect-SWP2-04.pdf+%22Weisner+and+Cronshaw%22+personnel+psych+meta-analytic&hl=en&gl=us&pid=bl&srcid=ADGEESgMV1M0js78u-t4Q6O_ynMFv62XuXHlwtTeRK9T7jLUu9pSrZ0NBHlO9nZl238Xsa5hBGymFFTez7J3RNgwBu0L2_LMkrpQlegaN-VmODiKM6aVTwLO_ePmD4_dFYKsqPFFV11s&sig=AHIEtbQthrI_ECUQU_vrCIiDdZaWW7ZgEg

This basically says that there’s a bias which causes interviewers to rate any given candidate better if there are previous poor candidates and worse if there are previous good candidates, which can produce some incorrect decisions. However, the authors says that you can minimize this through various steps.

Cheers,

Keith

Keith Wonderful suggestion. BTW, you had asked previously about whether there was evidence for improved accuracy for multiple interviewers. There is. See the Weisner and Cronshaw metaanalysis, 1988, Personnel Psychology.

Does this conversation have a half life?

IMHO, there’s too much opinion being paraded about as fact here on ERE.

-Keith

*Please hold me to this standard, too. -kh

To your comment about “do[ing] the math”, wasn’t it you that gave me the business (no pun intended :) ) with the following comment regarding QOH:

“Regarding Joshua’s cost of capital comment, one could gain some brownie points with the financial crowd by citing Net Present Value, but it won’t make much difference . . . ” (http://bit.ly/9sWvrL)

I’m glad to see you have come around to the importance of financial justification. As for me, I still feel the same today as I did then – if you can’t speak CFO-style, you can’t sell internally.

However (at least to me), “do[ing] the math means going beyond ROI”. Andrew Rudin agrees with the futility of ROI – http://bit.ly/aEGTDc

Your friend,
Tom (who needs enemies when they have Tom for a friend) Janz,
PS Did you take a peek at the Strengths Inventory?

Email me and I’ll make you a panelist (however, I have mike control)

It would be great to find other recruiters who, like Lou, actually want to earn a place at the C-Table by being able to “do the math”, instead of just whining about it in twits (Oh–did I spell that wrong?).

It’s validity is increased since we urge teams to debrief collectively and not make a yes/no decision until the team is in close agreement on all ten factors. We also suggest that as part of the training that interviewers grade themselves on each of the ten factors and then prove their rankings to someone else. This is a great learning exercise on the need for evidence rather than intuition to make an assessment. You can download a sample form from our website to try this out. You can also audit module one of recruiter boot camp online if you’d like to gain a sense of this.

Keith “Likes Things to Be Properly Named and Will Look on The Site To See What’s What” Halperin

Lou’s ratings, based on carefully guided professional judgment can be much more valuable for getting at true performance differences than a set of objective numbers contaminated by luck or bias.

And Lou’s use of assessment testing to back it all up—that’s my kind of process where multiple perspectives of potential AND performance lead to a more complete picture.

All of these assessments are based on our structured performance-based/behavioral interview process which is fully described in my book, Hire With Your Head and validated by OD and legal. (This was for Tom’s benefit)

Cheers,
Keith “Tries Not To Mix Apples With Oranges” Halperin

Regardless, your point that compensation matters as part of a raising the talent bar, is absolutely valid.

I’m just having fun – this is a good conversation, seriously :) I like to inject a little humor when things get a little hot in the kitchen, but I’m learning from everyone here :)

Below is the first of four performance objectives for a Sr. Product Manager. This is what we use when we initially take an assignment. We then rank all candidates on a 1-5 scale with respect to how well they’ll be able to deliver the objectives to determine what third they’ll be in. This is how we ensure candidates will be in the top-third of the existing workforce.

We also will consider candidates even when they don’t have the required experiences listed on the job description. This opens up the pool to more highly qualified candidates – including more diverse candidates. By looking for career gaps in comparison to this performance profile during the interview we can then present the job as a career move (sorry Joshua – I haven’t had your $$ problem, since I always present jobs as career moves this way.) Comp does become a problem when the job is a lateral transfer – which is a fundamental problem of using job descriptions.

Performance Profile for Product Marketing Manager

1. Conduct a comprehensive review of all new product programs: During the first 30 days prepare an analysis of all new product programs. Include status of product development effort, budget vs. forecast comparison, status of launch efforts, and major challenges and problems. Submit recommendations and plans to prioritize scarce resources to meet critical target dates, including resource needs (people, financial, capital).

2. Coordinate the development and launch of all the new product lines for the current season: Work with engineering, manufacturing, and marketing to determine status of all products to be launched this season. Identify hurdles and constraints and develop work-arounds as necessary to meet plan dates.

3. Lead the development of the two-year product plan: Take the lead on preparing the two-year product program due within 120 days, coordinating with product development and engineering. This needs to consist of competitive analysis, assessment of technology trends, and market and consumer research. Include revenue and market share analysis by channel, including segmentation analysis. New products (introduced within 12 months) should represent 20% of total revenue within 18 months.

4. Conduct a process review of the product development process: New products typically miss initial projections from a time and budget standpoint. Assess the product development process from all aspects and prepare a plan-of-action to re-organize all aspects of this effort. The goal is to have a predictable new product introduction program within 12 months.

The results we’ve had with this process is exactly as described in Gallup’s huge study on employee performance and satisfaction described in First, Break All of the Rules. Candidates hired this way perform at peak levels, they have increased job satisfaction and lower turnover. (Tom – is their work scientific enough?) In this study it’s clear that their 12 core criteria that increase personal motivation are correct. The first is clarifying job expectations upfront. NONE of the 12 have to do with having the requisite skills and experiences.

Anyway, I started with a premise – “It’s all about the compensation.” After diving and diving and diving (and diving some more), I returned to square one again:

“If you want to hire great, then pay great. Period.”

If you pay garbage, you recruit garbage. Too many companies want to recruit top talent at bottom barrel prices. Sure, maybe you catch an A-hire anomaly in the bunch (because they live 2 miles down the street), but those individuals are not the norm. The only exceptions I can find are non-profits making a difference in the world . . . and the U.S. Marines (they sell culture and inclusion, nothing more, nothing less. They sell the opportunity to “be one of us, the Few, the Proud.”)

I met a ton of people in grad school who worked for bummish organizations . . . and they knew it. But the bummish org was paying for their MBA (so they were buying the talent for a couple more years), but above all, the talent hung around at the bummish org because they paid the big bucks.

Fast forward to today. As an Exec Recruiter, guess who’s the toughest person to recruit? Most ERE members know, but if you don’t know, I’ll tell you – the talent already making the big bucks. You want to hire and retain A-level talent? Then be willing to pay for it and leave behind any false realities of slave labor among a world of unicorns and minotaurs.

Lou, who actually are “the best”? If you refer to quantifiable revenue-producers, that’s easy to define, and I’m a strong proponent of hiring the very best sales reps you can and treating them very, very well. What about those that either aren’t revenue producers or aren’t quantifiable? How do you rank people in these categories?
While my temperament would like to see everything/everyone all nicely classified, sorted, and ranked, with most kinds of jobs, it would be very hard to do.

Cheers,
Keith “Running Low on Glib Remarks” Halperin