I saw this story this morning, about IBM looking for more markets for its Watson information-sifting system (the one that performed so publicly on "Jeopardy". And this caught my eye for sure:
John Baldoni, senior vice president for technology and science at GlaxoSmithKline, got in touch with I.B.M. shortly after watching Watson’s “Jeopardy” triumph. He was struck that Watson frequently had the right answer, he said, “but what really impressed me was that it so quickly sifted out so many wrong answers.”
That is a huge challenge in drug discovery, which amounts to making a high-stakes bet, over years of testing, on the success of a chemical compound. The failure rate is high. Improving the odds, Mr. Baldoni said, could have a huge payoff economically and medically.
Glaxo and I.B.M. researchers put Watson through a test run. They fed it all the literature on malaria, known anti-malarial drugs and other chemical compounds. Watson correctly identified known anti-malarial drugs, and suggested 15 other compounds as potential drugs to combat malaria. The two companies are now discussing other projects.
“It doesn’t just answer questions, it encourages you to think more widely,” said Catherine E. Peishoff, vice president for computational and structural chemistry at Glaxo. “It essentially says, ‘Look over here, think about this.’ That’s one of the exciting things about this technology.”
Now, without seeing some structures and naming some names, it's completely impossible to say how valuable the Watson suggestions were. But I would very much like to know on what basis these other compounds were suggested: structural similarity? Mechanisms in common? Mechanisms that are in the same pathway, but hadn't been specifically looked at for malaria? Something else entirely? Unfortunately, we're probably not going to be able to find out, unless GSK is forthcoming with more details.
Eventually, there's coing to be another, somewhat more disturbing answer to that "what basis?" question. As this Slate article says, we could well get to the point where such systems make discoveries or correlations that are correct, but beyond our ability to figure out. Watson is most certainly not there yet. I don't think anything is, or is really all that close. But that doesn't mean it won't happen.
For a look at what this might be like, see Ted Chiang's story "Catching Crumbs From the Table", which appeared first in Nature, and then in his collection Stories of Your Life and Others
, which I highly recommend, as "The Evolution of Human Science".
1. patentgeek on February 28, 2013 3:19 PM writes...
Eventually, AIs will progress to proposing de novo novel compounds for evaluation, and some will be successful. I hope I'm around for the gnarly questions on inventorship under US law that will occur when that happens. "...beyond our ability to figure out." Heh-heh.
Derek's recommendation re Ted Chiang is strongly recommended!! His "The Merchant and the Alchemist's Gate" is a modern-day Arabian Nights tour de force.
Permalink to Comment2. josh bloom on February 28, 2013 3:47 PM writes...
I'll take "Lipinski Violators" for 400, Alex.
Permalink to Comment3. oldtimer on February 28, 2013 4:10 PM writes...
Hmmm, teams of bright PhDs with tons of experience, and insight wade through a great deal of poorly validated (clinically) biological data and progress compounds to the clinic where for the most part they fail. What makes you think that Watson or any other expert system will do better? It is not the sifting of the data that is the issue, it is the quality of and validity of the data.
Permalink to Comment4. Foolery on February 28, 2013 4:22 PM writes...
There is another Slate article here: http://www.slate.com/articles/technology/robot_invasion/2011/09/robot_invasion_can_computers_replace_scientists_.html
The program, Eureqa, has actually come up with an analysis of cellular behavior that holds true and is currently unexplainable.
Permalink to Comment5. anon the II on February 28, 2013 4:23 PM writes...
This is a crock. First of all, if those pansies at IBM were real men, they would have made that stupid Watson parse the damn questions like the two humans had to do. The technology is certainly there for a camera to read the board, do the ICR and feed the answers into Watson's search engine. But Noooooo. They got the answer fed from a text file straight into Watson's feeble brain. So while Jennings and that other sap are trying to read the first character in the answer, Watson was already scanning the databanks and slamming the button. I was embarrassed for science to watch that crap and hear it called competition.
When the battle comes, I'm siding with the humans.
Permalink to Comment6. gippgig on February 28, 2013 4:32 PM writes...
This would probably work reasonably well for things drugs all have in common, such as finding compounds with low toxicity, good pharmacokinetics, etc. Predicting compounds with totally new modes of action would probably require much more detailed information on how cells function at the molecular level then is available. Maybe in a few decades...
Permalink to Comment7. Curious Wavefunction on February 28, 2013 4:41 PM writes...
Watson: I am sorry Dave, I cannot let you synthesize that compound.
Permalink to Comment8. Curious Wavefunction on February 28, 2013 4:42 PM writes...
Watson: I am sorry Dave, I cannot let you synthesize that compound.
Permalink to Comment9. Mad Dog on February 28, 2013 4:54 PM writes...
@7: Are you implying Watson is one of the Pfizer "designers"?
Permalink to Comment10. darwinsdog on February 28, 2013 4:56 PM writes...
The day a computer does anything more than tell the programmer the same thing they could have calculated themselves but just in a faster manner is the day I...hey wait...uh, Google what do you mean 'self aware'..I didn't ask...easy now...[EXTERMINATE !, EXTERMINATE !]
Permalink to Comment11. metaphysician on February 28, 2013 5:10 PM writes...
It may not be yet, but it is a legitimate matter of discussion for the future. I would say that it should only qualify as a "designer" for the purposes of patents if it also qualifies as a *person*, and is thus as much an employee of the company as any human. Of course, that is its own giant massive ball of wax. . .
Permalink to Comment12. josh bloom on February 28, 2013 5:10 PM writes...
@Curious-- VERY funny
Permalink to Comment13. matt on February 28, 2013 5:28 PM writes...
Yes, but can Watson avoid embarrassing gaffes or prove that overexposure to antioxidants is contributing to cancer rates?
Permalink to Comment14. ex-GSK on February 28, 2013 5:34 PM writes...
Bet Watson could shift through the Sirtris data and suggest the company was worth somewhat less than the $750M GSK spent for them.... But then, so did the due diligence team at GSK. You have to actually base decisions on data to care.
Permalink to Comment15. Anonymous on February 28, 2013 6:32 PM writes...
Well if John Baldoni thinks it's worth investigating, it must be good!
Permalink to Comment16. hibob on February 28, 2013 7:18 PM writes...
We've " been there" ever since genetic algorithms were introduced: completely impenetrable code and decision making. The jump is when the questions getting correctly answered are ones we have no understandable way of solving, as opposed to being merely difficult.
Permalink to Comment17. Insilicoconsulting on February 28, 2013 10:12 PM writes...
Look upon it as a very efficient form of search, given the right parameters(human experts), the right data and background knowledge .
It may outperform human intuition in terms of lesser false positives and negatives, but we would still be the ones defining the context and prior knowledge fed to the beast.
Permalink to Comment18. Anonymous on March 1, 2013 4:58 AM writes...
John Baldoni.......! A lot of people @GSK are rolling around the floor laughing! Perhaps the Great Baldoni will invest in a Watson and sack a few scientists to make room for it.
Permalink to Comment19. John Spevacek on March 1, 2013 8:55 AM writes...
I'm struck by the fact that this may soon create "obvious" solutions for the pharma industry, meaning no more patents - "It's obvious to ask Watson and then try what he suggests." After all, this is well within the capabilities of the iconic PHOSITA (Person Having Ordinary Skill In The Art)
Permalink to Comment20. TheGreatBaldoni on March 1, 2013 9:28 AM writes...
Would the real Baldoni please stand up?
Permalink to CommentIs it this one? http://www.baldoni.dk/
or this one? http://www.amazon.com/John-Baldoni/e/B001IXS1FO/ref=sr_ntt_srch_lnk_1?qid=1362148054&sr=1-1
21. anon on March 1, 2013 10:25 AM writes...
would all the junk research out there be included too? and would Watson know to ignore it?
Permalink to Comment22. Woos? on March 1, 2013 10:29 AM writes...
Let Watson seat at any GSK portfolio review meeting and he will perform like everybody else in term of probability of success.
Permalink to Comment23. watcher on March 1, 2013 10:57 AM writes...
My bet is the Watson found what any good chemist and biologist team would do when looking under the lamp post. Can it actually move the vision? Unlikely.
Permalink to Comment24. simpl on March 1, 2013 11:06 AM writes...
agree with 16
Permalink to CommentWe saw this with visual control of vial production. There was no way the first generation of machines could match human skill. But over thirty years, the makers have made enough progress that the job is now better done by machine.
25. jimrandomh on March 1, 2013 12:37 PM writes...
My guess is that the 15 candidate drugs it suggested were, in fact, mentioned as candidates by papers in the dataset. That is, in fact, a pretty good way of finding candidate drugs (as long as no one else got there first), but it'd be a case of Watson acting like the fancy search engine that it is, and not like the AI its PR claims.
Permalink to Comment26. Luis Pedro Coelho on March 1, 2013 1:29 PM writes...
Watson is probably well into the black box mode.
I think we need to start to become comfortable with the fact that computers can have amazing intuitions: correct inferences that are impossible to rationalize.
Permalink to Comment27. MoMo on March 1, 2013 3:18 PM writes...
Not impressed. They fed the scientific literature, primarily chemical and biological abstracts, and came up with only 15 new anti-malarials? One can do the same with Scifinder and use Tanimoto similarity searches to do the same thing. In about 15 minutes and using a laptop.
Good waste of time, Watson.
Permalink to Comment28. DCRogers on March 1, 2013 4:03 PM writes...
Beware of "uninterpretable" AI models.
I'm reminded of a story (perhaps apocryphal) of a university that wanted a filter for the mountains of applicants. They used Machine Learning to build a complex AI model that did as good a job sifting the candidates as did their human experts, in a fraction of the time.
Punch line: they got sued for sex discrimination. They said, hey, not possible, we're using AI! Belatedly looking into the complex model, they found to their chagrin logic something like:
IF (SEX EQ 'F') THEN
Permalink to CommentSCORE = SCORE - 10;
ENDIF;
29. metaphysician on March 2, 2013 10:05 AM writes...
#28-
Of course, that opens some very interesting questions: what if an objective, unbiased AI *does* turn up logic like that in its statistical assessments? Is it discrimination if it can be actuarially supported?
Permalink to Comment30. hibob on March 2, 2013 12:08 PM writes...
#29:
"Is it discrimination if it can be actuarially supported?"
It could still be discrimination if underperformance by women at the university was due to actions by that university as opposed to the nature of the applicants/students.
Legally allowed? Sex falls under intermediate scrutiny, but the Supreme Court normally puts it at the end of intermediate scrutiny that's closest to strict scrutiny.
Permalink to CommentProbably not.
31. A. Postdoc on March 4, 2013 11:33 AM writes...
A total crock. Science by press release is awful. Drawing attention to this diminishes all the efforts of people doing real virtual screening with actual followup testing. Derek, I think you should try to highlight some of those.
Permalink to Comment32. MIMD on March 5, 2013 1:06 PM writes...
See:
"To Save Everything, Click Here: The Folly of Technological Solutionism"
http://www.amazon.com/Save-Everything-Click-Here-Technological/dp/1610391381
Permalink to Comment