Charles Murtaugh has an interesting piece on drug discovery, and invites me to correct any misstatements that he may have made. Hey, under my no-sparrow-shall-fall policy on mentions of drug discovery in the blogosphere, I'd have put my oar in anyway.
Charles's main point is exactly right: that discoveries in the medical/biotech field are often way out in front of any understanding of why they work. To my mind, it comes down to the distinction between engineering and basic research. An engineering problem is one that's largely constrained by known limits; its solution is the best that can be achieved within those limits. Want to make a Mach 5 airplane, bridge the Bering Strait, shrink a chip by a factor of ten? Here are the materials that you can work with; here are their properties (and, lest we forget, here's what they cost!) There's room for huge amounts of ingenuity within those constraints, but if you try to get out of them, your work turns into a research project very quickly.
We already know that things can travel at Mach 5, that long stretches of water can be spanned, and that chips can be made very small. It's a question of whether a usable plane, bridge, or chip can be made to do what you want, with the materials you have. At some point, you'll run into the limits of those materials, and the closer you get to those limits, the more basic research gets mixed into the engineering. If you decide that you need something for your airplane wing that has the melting point of titanium and weighs less than aluminum, well, you're going to be at it a while.
The examples that Charles uses in his piece - Wright brothers, Microsoft, Intel - are actually engineering projects, for the most part, which is one reason they sound odd when contrasted with basic research in the medical field. That's because science lives, of course, by finding weird results that are outside current understanding. When something sufficiently unusual comes along, we all jump on it and try to figure out why it does that, what we can use it for, what else acts like that, what happens if you mess with this part over here. . .whoops!
In our corner of it, as Charles points out, you can know that a medicine works without knowing how it works. As far as the FDA goes, if you can prove safety and efficacy (Phase I and Phase II/III trials, respectively,) then you're in, mechanism or not. But even if you know a lot about the mechanism you usually have no idea if what you're trying to do is possible or not.
Will inhibiting angiotensin-converting enzyme lower blood pressure? Can you make a small molecule that will inhibit the enzyme? The answer to both of those is a resounding "yes," as has been established. OK, what if you inhibit that one and neutral endopeptidase - the rationale for trying this is just as reasonable. Can you make a molecule that does it, and will that combination work even better? The answer to the first question is a resounding "yes," again. And the answer to the second, as Bristol-Meyers Squibb found to their sorrow, is an exhausted, resigned "no, probably not."
Of course, knowing the molecular mechanism for a drug helps tremendously in developing it. Most big companies prefer to work on targets that they have such an understanding of, because it helps you set up high-throughput assays to find the good stuff faster. The most frustrating sort of project is one where you have some wonderful effect in an animal model, but no idea of why it happens. You can end up making compounds and taking directly into the animal model, which is the old-fashioned (and slower, and more costly, and harder to optimize) way of doing it.
To directly address a question that Charles asks, we do indeed spend a lot of time changing our lead molecules around to see if they work better. It's not quite random, though, since we have a legacy of well-known tricks that have been shown to help in other projects (and a legacy of well-known things to avoid, too.) But once we're inside a given series of compounds, it's true: we just make as many different ones as we can, and let the assays sort 'em out.
He's also right that the full-scale "rational drug design" approach has been a bust, so far. (I talked about this back on February 7.) We just don't know enough yet to get this to work the way it potentially could. And his point about nanotechnology is very well-taken indeed: the whole idea of medical nanotech is predicated on knowing a great deal about exactly what you're trying to accomplish and how to do it, and both those items, unfortunately, are often on long-term back order.
Will they arrive eventually? That's where I get into my odd pessimist/optimist thing again. Long term, I think we really will figure this stuff out. But it'll be a longer term than we'd like, with a lot of twists and turns. Even so, I still see all these problems as eventually yielding to human skill and ingenuity - great big heaps of skill and ingenuity, and piles of time and money. My optimism comes from the first half of that sentence; my pessimism from the second. They coexist, but on different time scales.