Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Avastin's Metastatic Breast Cancer Approval Revoked | Main | Avastin Coverage, Amended »

November 21, 2011

Of Drug Research and Moneyball

Email This Entry

Posted by Derek

This piece on Michael Lewis and Billy Beane is nice to read, even if you haven't read Moneyball. (And if you haven't, consider doing so - it's not perfect, but it's well worth the time). Several thoughts occurred to me while revisiting all this, some of them actually relevant to drug discovery.

First off, a quick peaen to Bill James. I read his Baseball Abstract books every year back in the 1980s, and found them exhilarating. And that's not just because I was following baseball closely. I was in grad school, and was up to my earlobes in day-to-day scientific research for the first time, and here was someone who applied the same worldview to a sport. Baseball had long been full of slogans and sayings, folk wisdom and beliefs, and James was willing to dig through the numbers to see which of these things were true and which weren't. His willingness to point out those latter cases, and the level of evidence he brought to those takedowns, was wonderful to see. I still have a lot of James' thoughts in my head; his books may well have changed my life a bit. I was already inclined that way, but his example of fearlessly questioning Stuff That Everybody Knows really strengthened my resolve to try to do the same.

A lot of people feel that way, I've found - there are James fans all over the place, people were were influenced the same way, at the same time, by the same books. It took a while for that attitude to penetrate the sport that those books were written about, though, as that article linked to above details. And its success once it did was part of a broader trend:

Innovation hurts. After Beane began using numbers to find players, the A’s’ scouts lost their lifelong purpose. In the movie, one of them protests to Pitt: “You are discarding what scouts have done for 150 years.” That was exactly right. Similar fates had been befalling all sorts of lesser-educated American men for years, though the process is more noticeable now than it was in 2003 when Moneyball first appeared. The book, Lewis agrees, is partly “about the intellectualisation of a previously not intellectual job. This has happened in other spheres of American life. I think the reason I saw the story so quickly is, this is exactly what happened on Wall Street while I was there. . .”

(That would be during the time of Liar's Poker, which still a fun and interesting book to read, although it describes a time that's much longer ago than the calendar would indicate). And I think that the point is a good one. I'd add that the process has also been driven by the availability of computing power. When you had to bash the numbers by hand, with a pencil, there was only so much you could do. Spreadsheets and statistical software, graphing programs and databases - these have allowed people to extract meaning from numbers without having to haul up every shovelful by hand. And it's given power to those people who are adept at extracting that meaning (or at least, to the people willing to act on their conclusions).

The article quotes Beane as saying that Lewis understood what he was doing within minutes: "You’re arbitraging the mispricing of baseball players". And I don't think that it can be put in fewer words: that's exactly what someone with a Wall Street background would make of it, and it's exactly right. Now to our own business. Can you think of an industry whose assets are mispriced more grievously, and more routinely, than drug research?

Think about it. All those preclinical programs that never quite work out. All those targets that don't turn out to be the right target when you get to Phase II. All those compounds that blow up in Phase III because of unexpected toxicity. By working on them, by putting time and effort and money into them, we're pricing them. And too much of the time, we're getting that price wrong, terribly wrong.

That's what struck me when I read Moneyball several years ago. The problem is, drug research is not baseball, circa 1985. We're already full of statisticians, computational wizards, and sharp-eyed people who are used to challenging the evidence and weighing the facts. And even with that, this is the state we're in. The history of drug research is one attempt after another to find some edge, some understanding, that can be used to correct that constant mispricing of our assets. What to do? If the salt has lost its savour, wherewith shall it be salted?

Comments (17) + TrackBacks (0) | Category: Business and Markets | Drug Industry History | Who Discovers and Why


COMMENTS

1. dearieme on November 21, 2011 10:31 AM writes...

"...Liar's Poker, which still a fun and interesting book to read": spot on - I reread it a fortnight ago and it stands up very well.

Permalink to Comment

2. Jonadab on November 21, 2011 10:45 AM writes...

The major difference I can think of is that baseball players were still being evaluated based on their past performance. Sure, the method of evaluating that past performance was improved by the use of statistics, but the question that was being answered remained, "How well has this player been playing?"

It's a question that's remarkably easier to answer than "How well is this player going to play in the future", although of course there is a correlation. You can use the former question as a proxy for the latter, but that only works as long as the players in question *have* been playing in the past. If you pick a bunch of kids nobody's ever put on the baseball field before, the same method won't work as well.

Permalink to Comment

3. Expat on November 21, 2011 11:31 AM writes...

I remember looking into a second career in licensing a few years ago and was a bit shocked to find that there was little emphasis on a potential candidate's scientific background or ability to evaluate in-licensing opportunities for their scientific merit. Rather, experience in "doing deals" or a "deal sheet" seemed to be the most valued criterion. I know that many in the field have very strong technical backgrounds, but overall this did not seem to be an over-riding factor in deciding what to license and how much to pay. Perhaps now would be a good time to rethink how in-licensing should be conducted?

Permalink to Comment

4. Cellbio on November 21, 2011 1:03 PM writes...

another major difference is that a statistical analysis for finding contributors to a baseball team could work when a great hitter has a "success rate" of 1/3 or less, and an average hitter is about 1/4. Great teams win at a rate of about 6/10, and the worse teams win at about 4/10. Fine tuning methodologies with these small differences could make for better averages, and put a better team on the field, improving revenue with lower costs. But did moneyball ever win a world series? I don't think it did, but some argue the methodology contributed to the Red Sox.

In any case, in Pharma, we don't get ticket and hot dog revenue for our efforts prior to our world series wins, and modest improvements in our batting averages (screening efficiency, other work flow improvements) or win percentage (achieving portal metrics, upping Ph1 entrants) have only associated costs prior to market success (smallcos might be able to realize gain here). In fact, as many have argued here, gaming this system and working hard to "value" these efforts seems only to stifle innovation and reward individuals who meet the internal metrics and play by the rules.

So, I don't really think our assets are mispriced, or at least don't see that as the big issue. Instead, the parallel I like better is that we value the wrong players, those that operate well in the system of drug discovery today, those that survive in political environments, and we actively work against innovators and the innovative process, in part by demanding to know more about future performance of molecules than is knowable.

Permalink to Comment

5. You're Pfizered on November 21, 2011 1:26 PM writes...

Baseball had Moneyball, we had Combinatorial Chemistry.

Sabermetrics has its value, but as was stated above, can't necessarily predict how a player is going to perform on any given year. It can help spot trends or potential, but it can't take into account outside influences that could affect one's statistics. A nasty divorce, a child being seriously ill, someone not training as hard in the off-season after signing a big contract, etc...

Beane is living off of his legend at this point. He is extremely over-rated as a GM at this stage of the game, IMHO.

Permalink to Comment

6. jamjel on November 21, 2011 2:02 PM writes...

"The history of drug research is one attempt after another to find some edge, some understanding, that can be used to correct that constant mispricing of our assets. What to do? If the salt has lost its savour, wherewith shall it be salted?"

I think there is such an edge, such an understanding, and, speaking as a relatively naive and uninformed outsider, it seems to me to be severely underappreciated and seldom employed.

Namely, the natural history of living things and their interactions as seen from an evolutionary point of view, focused specifically on the chemicals they deploy to effect their strategies.

For example, we are apparently in the midst of a recrudescence of bedbugs, and beyond the revulsion we all feel, there are some very intriguing aspects of their adaptation to feeding on us that may well have pharmaceutical significance, in my opinion.

Despite lots of looking, no one has yet found a pathogen they transmit. This is in stark contrast to other blood sucking insects such as fleas, mosquitoes, and assassin bugs, and is particularly surprising given how easily they seem to spread.

It's almost as if doing as little harm as possible is a part of their arsenal, and that might imply that they are highly adapted to us and aim for long term infestations as a default, so that keeping us in good shape is in their best interests.

That doesn't mean that they aren't manipulating us, however.

The redness and swelling some people develop at bite sites show they are injecting something. Bloodsuckers are known to favor anti-coagulants, but that may not be all in the case of bedbugs.

A bedbug would seem to derive unqualified benefit from rapid and deep sleep on the part of their intended victims, and a bedbug which could somehow make this more likely would have a very strong selective advantage over bedbugs which couldn't.

So if any organism has developed a minimally harmful chemical capable of putting us to sleep, I'd think it might be bedbugs, and that we should look at them as a potential source of sleep aids.

Permalink to Comment

7. Bill on November 21, 2011 2:40 PM writes...

This post is very timely for me. I have been on a Michael Lewis kick reading a handful of his books including Moneyball and Liar's Poker.

I agree with the above posts that Sabermetrics is not applicable to pharma. In fact, one of the lessons from Moneyball is to withhold judging a recruit until they have played college ball and accumulated reliable statistics. Prior to Billy Beane, too many scouts were judging a high school recruit by his size, swing, and speed without regard to stats. This is analogous to judging a compound's value without having in vivo human data. Unfortunately, pharmaceutical companies don't have reliable statistics until phase III trials, which doesn't help with the original problem.

@4 There definitely is a mispricing problem. Two examples that jump out are Sirna and Sirtris. Every scientist knew that Merck and GSK were getting ripped off. I think most of the mispricing comes from focusing on possible future market size. An exec starts counting the zeros after the dollar sign and ignores the underlying science.

If Billy Beane focused his recruiting on on-base percentage, I think pharmaceutical companies should focus on limiting side-effects. Side effects are normally the reason most drugs fail clinical trials. If the drug can get through clinical trials should we really care if the drug brings in $10 million or $10 billion, it's better than zero? Now the question is how can we get these statistics earlier than phase III?

Permalink to Comment

8. MTK on November 21, 2011 2:46 PM writes...

@you're pfizered and @2,

I don't think that sabremetrics attempts to predict future performance, per se. In fact, one of the tenets is that future performance can be somewhat reliably predicted based on past performance if the right measures and factors (age, park effects, etc.) are added in. Rather the point of sabremetrics,IMO, is to use alternative verifiably better measures to value performance. Basically, it tries to identify those statistical measures that materially effect the outcome of a game or a season as opposed to those stats that don't translate well into wins. The classic example being batting average as opposed to slugging percentage with the former having a much lower correlation with wins and losses versus the latter.

What Derek is saying is that if we apply that to drug discovery we need better measures of value for drug candidates. In some ways that's what the oft maligned Lapinski guidelines tried to do, isn't it? Of course, maybe that's analogous to batting average and what we need is something more like on-base % and slugging % to be truly better to assess value.

Permalink to Comment

9. bbooooooya on November 21, 2011 3:12 PM writes...

"@4 There definitely is a mispricing problem."

Today's $11 billion VRUS acquisition.....

LOL

Permalink to Comment

10. Morten G on November 21, 2011 3:51 PM writes...

Statistics + drug industry makes me think of Lipinski. Thinking of Lipinski makes me angry. Just Read The Goddamn Paper. If you made everyone in pharma read that damned thing (especially anyone in a mangement role) you would have come a long way.
Hell I'll summarize it: out of the ORALLY active drugs on the market in 1997, 2/3 of them only violated 1 out 4 rules imposing a limit on their complexity and hydrophilicity.
Anyone disagree then correct me. 6 years since I read it.

Permalink to Comment

11. Cellbio on November 21, 2011 4:53 PM writes...

To clarify, I definitely think assets are mispriced in deals, as noted above, but I took Derek's comment to mean something else altogether. I meant only that our "pricing" of pipeline assets through the time and effort we apply is not a central problem, more it is the culture and processes of pharma that are the core problem, which includes looking for the bigger fool. We must invest in research, without knowing we have created value, until late in our game. Trying to refine discovery efforts through business process efficiency exercises has been quite disruptive. The silly deals that get done, or overpaying for a company don't change this fundamental failing of pharma internal effort in my opinion.

Permalink to Comment

12. luysii on November 21, 2011 10:43 PM writes...

Drug discovery is so hard because, unlike baseball, we don't fully understand the system we're trying to affect (cellular biology, molecular biology, organismal biology). My guess is that we're not even close to understanding it. We don't know the targets we want to hit, and we don't fully understand what the drugs we have are actually doing.

I've made a new category on my blog which presently contains 18 articles detailing our ignorance of various aspects of this, and what we do in fact know. To read them you need to be a chemist, but the molecular biology you need is all there in 5 articles in another category https://luysii.wordpress.com/category/molecular-biology-survival-guide/

For a summary of the first 10 articles (and eventually all 18) see https://luysii.wordpress.com/2011/11/21/a-new-category/

Permalink to Comment

13. MLBpitcher_and_MedicinalChemist on November 21, 2011 11:22 PM writes...

Wow, an post about baseball from a baseball pitcher.

Permalink to Comment

14. drug_hunter on November 23, 2011 6:41 AM writes...

Luysii has it exactly correct. To cast it in Taleb's "Black Swan" terms, baseball is a "Mediocristan" problem while drug design is an "Extremistan" problem. If you know the history of a baseball player, you can do a really good job of predicting what he'll do next. The infield shift works for Ortiz, but wouldn't work for Ichiro (the classic spray hitter).

Now, a really interesting question is whether we will EVER be able to collect enough information for drug design to become truly predictive. Theoretically, yes. But to follow up on Derek's sentiments, I ask, "How long, O Lord?"

Permalink to Comment

15. mike on November 23, 2011 9:00 AM writes...

"We're already full of statisticians, computational wizards, and sharp-eyed people who are used to challenging the evidence and weighing the facts"

Although this is true, it isn't exactly the same. It is reasonably easy to evaluate a single baseball player's history and deduce how well he might do this year, like it is easy to look at a single drug's past sales and predict their present sales. But how does this apply to research? And where in baseball would you get the theory that, since you evaluated 50 walk-on players to find one promising player, all you would need to do is sample 250 random people to find another 5, which is the philosophy that drove so much of our combinatorial chemistry for so long.

Permalink to Comment

16. SweetPea on November 25, 2011 5:01 AM writes...

"Anyone in baseball could have pinched James's ideas".
Look at the reasons why Billy Beane spotted what Big Baseball didn't:

* the A's had little money
* they were located in an area rich in (non-baseball) ideas and innovation
* Beane himself, was an autodidact - hungry and agitated about new things

I can't see any of these applying currently to Big Pharma.


Permalink to Comment

17. Robur on November 25, 2011 5:17 AM writes...

The background to Billy Beane is illuminating: "denied the conventional college path (he) had that hunger. It's a huge advantage to him that he has some slight anxiety". (Contrast this with the "incurious people (who) will go to Princeton, Harvard, Yale, Stanford and come out and think they know everything").

For many years now, Pharma has had the money to recruit "the best" people from the best Universities.... maybe it was a mistake to stop asking about other qualities, such as curiousity and character?

Permalink to Comment

POST A COMMENT




Remember Me?



EMAIL THIS ENTRY TO A FRIEND

Email this entry to:

Your email address:

Message (optional):




RELATED ENTRIES
XKCD on Protein Folding
The 2014 Chemistry Nobel: Beating the Diffraction Limit
German Pharma, Or What's Left of It
Sunesis Fails with Vosaroxin
A New Way to Estimate a Compound's Chances?
Meinwald Honored
Molecular Biology Turns Into Chemistry
Speaking at Northeastern