There's a good article by Leandro Herrero in the October issue of Scrip magazine (no online content without a subscription.) He's teeing off on the overuse of numerical measures in the drug industry (and industry in general:)

"Your business education and experience tell you that if you can't measure, you can't manage. . .good managers are sometimes defined by their ability to measure outcomes; bad ones by the apparent vagueness in describing what needs to be achieved."

The way I've heard that first point expressed is "Anything that's measured will be managed." The problem is that many of the measurements are bogus. The urge to quantify things gets the better of us, and we attach numbers to things that either aren't measured well or can't be measured at all.

"A friend of mine once issued an open invitation to dinner for anybody who could unequivocally prove that any performance forecast by strategic planning (be it peak sales, market share, or NPV achievement) had ever been hit. To my knowledge he has not dined with anyone yet."

I wouldn't get a meal out of him myself. Over the years, I've seen some real where'd-that-number-come-from assessments of a compound's potential market. And, as Herroro points out, it does not breed confidence when the marketing gurus suddenly come up with a completely different number, carved into the same stone tablets as the first one. The higher the pressure to come up with an estimate, the less reliable it is. I've long summarized that effect as "You need a number real bad? Well, here's a real bad number!"

"In many areas of management today, many things won't be taken seriously (ie, qualify for funding) if their return on investment (ROI) can't be measured. . . never mind that the number has been cooked up via a combination of clever arithmetic. . .I have seen ridiculous ROIs that have gone down well with management, just because they were done, called ROI, and looked good."

There's the key point right there. People are going to believe what they want to believe, or what they feel that they should believe. Science gets us away from that a little bit, but it's still done by human beings, and these are human tendencies.

I can think of another area where a false attachment to measurement gets people into trouble. Drug companies often have an unhealthy fixation with quantifying their drug pipelines. And once you get serious about counting these things, it's impossible to resist the temptation to set them up as goals: next year, we'll start (insert whole number) projects and recommend development of (insert smaller whole number) clinical candidates. When you go down that road a little further, you come to another hard-to-resist temptation: starting a project (or worse, recommending something to the clinic) mainly just to make your numbers. Those of you working for Big Pharma companies are probably grinning sheepishly right about now. . .

I know, that isn't what's supposed to happen. It's not what anyone set out to do. But it's not where you start with this kind of thing: it's where you end up that counts.