Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Up Close and Personal | Main | Awash in Yen »

May 27, 2008

An Eye For the Numbers

Email This Entry

Posted by Derek

My wife and I were talking over dinner the other night – she’d seen some interview with the owner of a personal data protection service, and he made the pitch for his company by saying something about how out of (say) a million customers, only one hundred had ever reported any attempts on their credit information or the like. And my wife, who spent many years in the lab, waiting for what seemed to her to be the obvious follow-up question: How many people out of a million that didn’t subscribe to this guy’s service report such problems?

But (to her frustration) that question was never asked. We speculated about the reasons for that, partly out of interest and partly as a learning experience for our two children, who were at the table with us. We first explained to them that both of us, since we’d done a lot of scientific experiments, always wanted to see some control-group data before we made up our minds about anything – and in fact, in many cases it was impossible to make up one’s mind without it.

After a brief excursion to talk about the likely backgrounds and competencies of news readers on TV, we then went on to say that looking for a control set isn’t what you could call a universal habit of mind, although it's a useful one to have. You don’t have to have scientific training to think that way (although it sure helps), but anyone with a good eye for business and finance asks similar questions. And as we told the kids, both of us had also seen (on the flip side) particularly lousy scientists who kept charging ahead without good controls. Still, the overlap with a science and engineering background is pretty good.

What I’ve wondered, since that night is how many people, watching that same show, had the same question. That would be a reasonable way to determine how many of them have the first qualification for analyzing the data that come their way. And I’m just not sure what the percentage would be, for several reasons. For one thing, I’ve been working in the lab for years now, so such thinking is second nature to me. And for another, I’ve been surrounded for an equal number of years, by colleagues and friends who tend to have science backgrounds themselves, so it’s not like my data set is representative of the population at large.

So I’d be interested in what the readership thinks, not that the readership around here is any representative slice of the general population, either. But in your experience, how prevalent do you think that analytical frame of mind is? The attitude I’m talking about is the one that when confronted with some odd item in the news, says “Hmm, I wonder if that's true? Have I got enough information to decide?" It's an essential part of being a scientist, but if you're not. . .?

Comments (32) + TrackBacks (0) | Category: General Scientific News | Who Discovers and Why


COMMENTS

1. kmd on May 27, 2008 8:21 AM writes...

Derek,

What you are describing happens to me every time I speak with members of my family (all non-scientists). They'll bring up some sensationalized statistic from a magazine, news report, or tv advertisement, and remark at how amazing this fact is without consideration of a control. An example from last weekend is that my father mentioned how he had seen some news story about a kidnapping, and then followed up with "these things never happened when I was growing up..." Really? Or perhaps such events weren't publicized as aggressively then as they are in today's media? Or maybe you didn't notice then because you didn't have kids of your own? And how many kidnapping stories have you heard this year anyway, two or three? How many kids are there in this state, millions? Should you be that shocked? I received a blank stare from across the table...

Permalink to Comment

2. MiddleO'Nowhere on May 27, 2008 8:40 AM writes...

Your post reminded me of this comic.

http://xkcd.com/242/

Permalink to Comment

3. HelicalZz on May 27, 2008 9:28 AM writes...

I'm more cynical than you. If I don't hear a 'compared to what' when being sold based on a statistic, I assume I am being artfully misled.

Permalink to Comment

4. LNT on May 27, 2008 9:42 AM writes...

I think the public is maybe a little more tuned into "control" experiments than you guys give them credit for. Remember the old Tide commercials? (or any detergent for that matter) They always compare Tide to "detergent X" and show that the clothes washed with Tide come out whiter. Obviously that isn't a good control, but it demonstrates that the public at least has a partial grasp on the need for control experiments.

Permalink to Comment

5. Analytical Scientist on May 27, 2008 9:59 AM writes...

I think the analytical frame of mind is quite rare outside of the sciences. I'm shocked by how much ridiculous balderdash slips by unchallenged in the media, and how easy it is for the Kevin Trudeau's of the world to get rich on the backs of the naiive public. I often point these issues out to my non-scientist wife, but I suspect that she only listens because she's my wife.

Permalink to Comment

6. Sleepless in SSF on May 27, 2008 10:26 AM writes...

I guess I am about to reveal the depths of my misanthropy here, but what the hell. I view the continued existence of sp*m as absolute proof that far too many people are too stupid to manage their own affairs. Has anyone reading this blog ever been tempted to respond to any piece of sp*m, ever? I am betting not. And yet, sp*m persists. I know the argument that huge volumes are sent, and that it only takes a small response rate to keep it going, but I remain unconvinced.

I think the response rate to the entire corpus of sp*m is likely to be higher than any of us expect, and that for every response there are several cretins who think, "Yeah, I'd like to get me some of that" but don't act. I base this on the enormous and continually growing volume of the stuff; there is money being made there, and lots of it. If there weren't, sp*mmers wouldn't go to such lengths to continue to operate in the face of our best technical attempts to prevent them from doing so.

And to tie this to Derek's question, it is my further belief that anyone who responds to sp*m -- or even thinks about doing so for a fraction of a second -- would never think critically about the quality of the evidence presented to them for any proposition.

Permalink to Comment

7. John Spevacek on May 27, 2008 10:27 AM writes...

I'm always a big advocate that one number by itself is meaningless (a point already made in part by HelicalZz). Sometimes (not very often) the second number can remain unstated (i.e., if Senator Windbag introduces a new bill to create a new program and he needs $50,000 to do it, nobody would think that it is an expensive program) but it never hurts to make it explicit.

You can see this also in the point that kmd made. 2 kidnappings is not much given the millinos of kids in the state. Again, a second number is always needed to give the first number some relevancy.

Permalink to Comment

8. Sili on May 27, 2008 11:05 AM writes...

Mark Liberman voiced his opinion on this a while back:
http://itre.cis.upenn.edu/~myl/languagelog/archives/004992.html

And of course Ben Goldacre is desperately trying to do just that education of the general public that Liberman sighs for (Liberman does a good job of doing that on LanguageLog, himself, but he doesn't have a column in the Grauniad so to some extent he, too, is preaching to the choir):
http://www.badscience.net/?cat=7

I have to say, though, that just having a course in statistics, didn't do it for me. Of course, I've since come to realise that I'm not cut out for academia, so perhaps it's no surprise that despite quite a few years at uni, that necessary critical mindset didn't come to me naturally. In fact, I think I've become far more sceptical in the last few years after my depression incapacitated me and I started living online. That, I think, is testament to the value of blogs.

Permalink to Comment

9. TNC on May 27, 2008 11:10 AM writes...

I'm a chemist and my wife is an ICU RN. Recently, I told her that we were among the less than 20 million US households that didn't have cable TV; immediately, she said that to fully understand the statistic, she needed to know how many households there were in the US.

I'd like to think this is because she hangs out with me, but I think it's just 'cause she's smart.

Permalink to Comment

10. vasili on May 27, 2008 11:13 AM writes...

Maybe, she hangs out with you because she is smart.

Permalink to Comment

11. CMC guy on May 27, 2008 11:31 AM writes...

Didn't you ever do Myers-Briggs personality/organizational typing in any of your various management fads? I don't have the numbers handy but recall "analytical" dominate thinking was something like 10-20% in general population but >50% among scientist (I was surprised not higher although believe categorization does not include people who have more flexible situation approaches and use analytical often). Besides becoming scientists there where a few other professions such analytical thinkers gravitated too that I don’t remember specifically (Engineers & Finance?). Whatever those numbers are I would agree this isn't how large majority react when they hear something that needs a fuller context for understanding.

I think other factors to consider when suggesting not common are the education system and media presentation both of which IMO run counter to principle of deeper evaluation. Although some teachers still aim to encourage kids to learn how to think critically there is so much straight regurgitation of facts as the measure of achievement. The media in general likewise spits out sound bites/headlines with frequent incomplete or unbalanced information especially if a complex issue. These things reinforce a laziness when it comes to asking pertinent questions.

Permalink to Comment

12. Nick K on May 27, 2008 11:36 AM writes...

The people who use logic and deduction in their everyday life, like car meechanics, electricians and plumbers, never have the slightest difficulty understanding how science works. Arts graduates, on the other hand, will never, ever grasp this. Unfortunately, the latter group dominate the media.

Permalink to Comment

13. Anon on May 27, 2008 11:37 AM writes...

The analytical frame of mind is quite rare, especially when it comes to politics...

Blogs such as Powerline and LGF frequently expose, for example, factual omissions and distortions in The Paper of Record (NYT). And, those blogs are authored by lawyers and artists/computer programmers, respectively.

Permalink to Comment

14. Chrispy on May 27, 2008 12:25 PM writes...

A recent NYT article suggests that LifeLock does, in fact, misrepresent the data in order to get more subscribers.

Permalink to Comment

15. Philip on May 27, 2008 12:26 PM writes...

As part of the “How People Think” discussion, I’m reminded of the current Democratic presidential race. My wife and I (both scientist) have been amazed at some of the arguments put forth by supporters of Clinton, on why she should still be in the race. Most of these people are quite intelligent and yet, they engage in the most illogical banter.

I think I’ve figured it out.

There are at least two types of people. There are scientists, who look at the facts and the rules and try to arrive at the truth. And there are lawyers, who take a position and try to select facts and rules that advocate for or support that position. You have to be intelligent to do either very well. I don’t know that you can be good at both. Unfortunately, I think the lawyers make more money.

Anyhow, when you realize that these TV talking heads are advocating and not rationalizing, it makes a bit more sense. The challenge is to not start believing that those lawyers who agree with you on this particular point are necessarily going to be logical when the next disagreement comes around.

Permalink to Comment

16. retread on May 27, 2008 12:44 PM writes...

Somewhat off topic, but which should be of interest anyway is the following.

Part of the neurological exam (and it should be part of the general physical exam) is the examination of mental status. It has several parts -- just about everyone checks for orientation (time, place, person) and notes the emotional state. Among the other parts of the mental status exam, budding neurologists are taught to test abstract thought (proverb interpretation -- e.g. what does "a stitch in time saves nine" mean, similarities and differences -- e.g. what is the same about a table and a chair, an apple and an orange, what is different about a river and a lake etc. etc.)

We were told that the inability to do this was abnormal. I gave it up as soon as I finished training. Too many people failed the tests, and they clearly weren't abnormal.

Proverbs weren't interpreted, but repeated back a bit louder or just not understood. (I always checked to make sure that the individual had heard the proverbs before).

Table and chair -- gave forth -- 4 legs, you sit on one and don't on the other (recall that simlarities rather than differences were asked for). Furniture wasn't mentioned

Apple and Orange -- the different colors were named (not the fact that they were both fruit) or they are both round -- partially right.

I would guess that between 25% and 50% of the non-academic and non-scientific general population answers this way.

So, in my experience, a large percentage of the populace thinks extremely concretely and non-abstractly. Try it on a few acquaintences -- guys pumping gas, supermarket checkers, waitresses or any other nonacademic types you care to -- I think you'll be surprised.

With regard to Nick K "The people who use logic and deduction in their everyday life, like car mechanics, electricians and plumbers, never have the slightest difficulty understanding how science works." - However, in my experience they didn't do particularly well on the above tests.

Check it out.

Retread

Permalink to Comment

17. Betsy on May 27, 2008 12:45 PM writes...

When I was growing up, I was a good little girl and memorized what I learned in school and got by. When I got to college, I was taught to question everything, and it was as if the whole world opened up to me. I don't think we encourage our kids enough to question what they're learning, to consider the source, and make their own judgments. The result is a general public that doesn't think analytically. I think some people come by it naturally, but most need to be encouraged to think that way.

A few months ago I was going over a protocol with one of our RAs. I was trying to explain why I thought she wasn't setting up the experiment properly and we got in a big argument. She finally understood what I was trying to say, and seemed really bothered that she hadn't thought of it before herself. She admitted that she just blindly followed the protocol, without questioning whether it made sense. She went on to say that she was never taught to question what she was told. Maybe some of this is cultural (she's from India), but she went to undergrad in the US. Is this how we're training our scientists?

What I find most disturbing is when I encounter fellow Ph.D. scientists who don't do proper controls, and then make broad, sweeping conclusions about their meaningless data.

Permalink to Comment

18. SRC on May 27, 2008 12:54 PM writes...

My long-suffering wife has to listen to me rant when I encounter logical flaws in media reports (you can imagine...).

My favorites are silly statements such as "breast-fed babies have higher IQs, so breast-feed your baby!" Gee, any other correlates there? Breast-fed babies are probably likely to be driven in Volvos and have trust funds, too, so if you want your kid to have a trust fund, breast-feed!

(Nothing against breast-feeding, mind you, just pointing out the silliness of an argument adduced in its favor.)

I oscillate between thinking, on one hand, that reporters are cynically writing this type of thing to sell papers, and on the other, that they're too dull to realize what nonsense they're writing. (I have the same oscialliation with respect to management fads.)

Permalink to Comment

19. Joe Bingham on May 27, 2008 2:52 PM writes...

The average NPR reporter is pretty educated, I think, but rarely a day goes by when I don't hear a reporter on All Things Considered or Marketplace spit out statistics that're meaningless without some sort of comparison. Since I'd guess that education tends to correlate with an analytical mindset, I'm guessing that most people don't have it (since it's so rare within the subset of people I'm guessing are most likely to have it).

Permalink to Comment

20. John Q Public on May 27, 2008 2:56 PM writes...

I want to see the control blog entry before I can intelligently comment on this one.

Permalink to Comment

21. cellbio on May 27, 2008 3:16 PM writes...

I think not too many people think through things, but rather come to a conclusion too quick, either pointed in that direction by the source of info (eg. politics) or by bias or experience. It is part of normal thought processes, top-down processing, that allows for quick decision making. Scientist often see details better rather than rushing to a synthesis. Scientists can also drive business types nuts by appearing to never come to the point, or synthesis of the details.

Here is a fun test: Set up someone by saying that a new study suggests workers fake sick days often. Continue by saying, the study found that 40% of the time workers were absent on a Friday or Monday, extending their weekend. Very few will point out the obvious fact that 2 days out a a five day work week is 40%.

Permalink to Comment

22. retread on May 27, 2008 3:23 PM writes...

SRC and all concerned:

Do not feel that superior (well you can to academic medicine if you wish ). The Women's Health Study was a disaster for some of the participants. For details see comment #2 on the 6 May post here about Alzheimer's disease. Over 50 relatively uncontrolled studies showed a benefit for hormone therapy in postmenopausal women prior to the first truly controlled trial.

Permalink to Comment

23. Jeb Bush on May 27, 2008 5:23 PM writes...

I am terrible in this regard and definitely have an analytical mind. A few years ago I read John Allen Paulos: A Mathematician Reads the Newspaper. Quite interesting. Currently I am reading a book about global warming and the flooding of America's coastal cities. It is almost laugh out loud funny since so many grand claims are made, but never any scientific references to back up anything. The other frustrating aspect of this is the simplification or politicization of any issue. Everything is boiled down to a black and white sound bite and truly complex issues are never really hashed out in an analytical manner.

Permalink to Comment

24. joel on May 27, 2008 7:13 PM writes...

hey, another scientist couple! based on my extensive study, I can conclusively say this is due to the fact that science and analytical thinking is dead sexy.

Wait, what kind of controls do I need to study that?

Permalink to Comment

25. SRC on May 27, 2008 9:30 PM writes...

Since I'd guess that education tends to correlate with an analytical mindset

Why would you think that? Liberal arts majors are anything but analytical. They're typically enchanted by novelty, even if it's utterly nonsensical. Novelty trumps common sense every time in the humanities, as far as I can tell.

Permalink to Comment

26. CMC guy on May 28, 2008 12:02 AM writes...

joel: My wife (Ag anniversary) also is a scientist (OK she's just a biochemist but we all have burdens to overcome ;)) and have known many such couples. Think those relationships have less to do with like interests/minds but more to limited easy opportunities to meet opposite sex outside the lab. When a sexy scientist calendar is published you can perhaps run a correlation to other standards (although not sure how to express analytical thinking part).

SRC; I have known many scientists/engineers who lack common sense of a broom stick so don't broad brush LA/Hum people. Anyway world would be rather dull and stagnant if didn't have varied personalities, perspectives and capabilities to interact with.

Permalink to Comment

27. Nick K on May 28, 2008 4:46 AM writes...

Retread: I'll try your suggestion sometime. My point was that plumbers, electricians etc are fully aware that there is an objective real world out there, and to derive any useful information about (eg finding and repairing a fault in a car) requires making hypotheses, testing them against physical reality, and discarding them if necessary. This is the essence of the scientific method. Arts grads are clueless in this regard.

Permalink to Comment

28. Nick K on May 28, 2008 5:11 AM writes...

I've just realised I made a sweeping generalisation unsupported by evidence in the above post. There must be plenty of Arts grads out there with anlytical and logical minds. I've just never met any.

Permalink to Comment

29. Retread on May 28, 2008 8:26 AM writes...

When I say normal people, I mean the cops and fireman and their families we took care of in Philly in the 60s when we were in training. These were important and functioning members of society effectively keeping the place from burning down, despite the concerted efforts of people who wanted it to.

Permalink to Comment

30. TW Andrews on May 28, 2008 9:59 AM writes...

Permalink to Comment

31. Darien on May 29, 2008 12:33 AM writes...

I'm not in the usual demographic of this web site -- I'm a blue-collar worker. Most of the people I work with have very limited education, and that usually trade school, so no real scientific background. Of this group, I think I've only had one person ever approach a claim with proper scientific skepticism, demanding evidence before he'd just accept it.

Right made my day when that happened, too.

Permalink to Comment

32. Haggis on May 29, 2008 6:52 AM writes...

We used to have many discussions about this sort of thing in Grad School. One observation was that there were very few Arts grads who had a good feel or interest for science but very often scientists had a very good feel for science AND the arts and were often very aware of arts, literature, music etc. Obviously it's a massive generalisation but I rather feel there's something in that. So does that me analytical types have broader minds?

Permalink to Comment

POST A COMMENT




Remember Me?



EMAIL THIS ENTRY TO A FRIEND

Email this entry to:

Your email address:

Message (optional):




RELATED ENTRIES
The Worst Seminar
Conference in Basel
Messed-Up Clinical Studies: A First-Hand Report
Pharma and Ebola
Lilly Steps In for AstraZeneca's Secretase Inhibitor
Update on Alnylam (And the Direction of Things to Come)
There Must Have Been Multiple Chances to Catch This
Weirdly, Tramadol Is Not a Natural Product After All