While I'm on the subject, there's another problem with employee rankings, one that doesn't just apply to research organizations. I first came across a statement of it while reading Bill James, who showed how it applies to baseball teams when they decide whether to bring in some veteran player to hold down a position or go with someone from triple-A. It's an over-reliance on normal distribution. (Here's a discussion of the idea.)

People have this mental picture of the classic bell curve - bulging middle, sloping down to the sides as it tails off to the few stars on the right, the few destructive losers on the left. In a departmental performance evaluation, you end up with most people getting "Meets Expectations" or the equivalent, some that rank higher, a few that rank lower. In my experience (two large drug companies,) most of the raw evaluations come back as "Meets" or higher, and it's rare that folks get initially ranked on the low end. That gets changed as the whole department comes into focus, though, because there's often this feeling that you have to rank some people low, in the same way that there have to be some star performers.

But here's the key: the performance ranking in any organization that is free to hire and fire its own employees will not fit a normal distribution. Why should it? A normal distribution is what you'd expect from a random sample, and I'll assume that most businesses don't hire or retain their employees at random. No, what you have is most likely the far right-hand side of a much larger distribution, the performance ratings of all the people you could have possibly hired for those positions.

One big factor that keeps things from being normally distributed is the entry barrier into a technical field. For the most part, you can't be stupefyingly incompetent and get a degree from a reasonably good research group at a reasonably good school, or be a total bozo and get in the door for a job interview with a decent resume and give a competent-sounding seminar. The total washouts are mostly gone by the time you place an ad in C&E News. Any of them that do send you resumes have a tougher time getting hired, and any of them that you actually hire have a tougher time being retained.

Another factor that bends the distribution is that people are actively trying to improve their rankings from year to year (well, at least some of them are.) No one's striving to slide down the list, that's for sure. The data points of a random sample aren't being told where they landed last time and given incentives to shift to the right.

So I'd say that a realistic batch of performance rankings has a majority at a "Meets Expectations" level, and the remainder stretching out toward the higher rankings. There really shouldn't be many "Below Expectations" people at all, because the whole point of a ranking like that is that they either shape up, or you ship them out.

This also points out the folly of the Jack Welch "rank 'em and yank 'em" style of performance review. You know, find your bottom 10% and fire them all. Other companies that have tried this technique (Ford, to pick a notable example) have found that it mostly sows fear and discord. And I'm sure it did at GE, too, truth be told (although some CEOs swear by it.)

The "bottom 10%" is basically identical to the 60 or 70% of your employees that are doing just fine, minus a few people having a bad review period (a different set each time,) and minus a few genuine losers. If you seriously try to fire this illusory bottom tier, you end up having to make arbitrary, meaningless distinctions on five-page HR forms in order to distinguish them. Find the real losers and heave them out, absolutely. But don't draw a ridiculous line in the sand and then jerk people around because of it. A really good manager should be able to fire someone without hiding behind a bad policy.