Corante

About this Author
DBL%20Hendrix%20small.png College chemistry, 1983

Derek Lowe The 2002 Model

Dbl%20new%20portrait%20B%26W.png After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases. To contact Derek email him directly: derekb.lowe@gmail.com Twitter: Dereklowe

Chemistry and Drug Data: Drugbank
Emolecules
ChemSpider
Chempedia Lab
Synthetic Pages
Organic Chemistry Portal
PubChem
Not Voodoo
DailyMed
Druglib
Clinicaltrials.gov

Chemistry and Pharma Blogs:
Org Prep Daily
The Haystack
Kilomentor
A New Merck, Reviewed
Liberal Arts Chemistry
Electron Pusher
All Things Metathesis
C&E News Blogs
Chemiotics II
Chemical Space
Noel O'Blog
In Vivo Blog
Terra Sigilatta
BBSRC/Douglas Kell
ChemBark
Realizations in Biostatistics
Chemjobber
Pharmalot
ChemSpider Blog
Pharmagossip
Med-Chemist
Organic Chem - Education & Industry
Pharma Strategy Blog
No Name No Slogan
Practical Fragments
SimBioSys
The Curious Wavefunction
Natural Product Man
Fragment Literature
Chemistry World Blog
Synthetic Nature
Chemistry Blog
Synthesizing Ideas
Business|Bytes|Genes|Molecules
Eye on FDA
Chemical Forums
Depth-First
Symyx Blog
Sceptical Chymist
Lamentations on Chemistry
Computational Organic Chemistry
Mining Drugs
Henry Rzepa


Science Blogs and News:
Bad Science
The Loom
Uncertain Principles
Fierce Biotech
Blogs for Industry
Omics! Omics!
Young Female Scientist
Notional Slurry
Nobel Intent
SciTech Daily
Science Blog
FuturePundit
Aetiology
Gene Expression (I)
Gene Expression (II)
Sciencebase
Pharyngula
Adventures in Ethics and Science
Transterrestrial Musings
Slashdot Science
Cosmic Variance
Biology News Net


Medical Blogs
DB's Medical Rants
Science-Based Medicine
GruntDoc
Respectful Insolence
Diabetes Mine


Economics and Business
Marginal Revolution
The Volokh Conspiracy
Knowledge Problem


Politics / Current Events
Virginia Postrel
Instapundit
Belmont Club
Mickey Kaus


Belles Lettres
Uncouth Reflections
Arts and Letters Daily
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline

In the Pipeline

« Depressing Figures for Acomplia | Main | Holiday Break »

November 20, 2007

And It Goes Like This!

Email This Entry

Posted by Derek

I had a hard drive failure the other day, which naturally got me to thinking about backing up data, and about the times I’ve been more paranoid about it. I wrote my PhD dissertation back in those far-off days (1988) when you could put Mac versions of Word and Chem-Draw on one 3.5-inch disk (yes, that was possible, and I still have the disk to prove it). But I went to the disk-swapping trouble of putting my dissertation-in-progress on a separate floppy.

So there I was, with a couple of week’s worth of dissertation draft on my floppy disk, when one fine day I insert the thing into the slot, and. . .it can’t be read. Hrm. I try other machines. I try them all. None of them can read the disk, under any conditions. It slowly dawns on me that my two weeks of work have evaporated, and a little later it dawns on me that things could have been much, much worse. I converted to the Backup Religion.

Grad students writing up tend to get a bit paranoid under the best conditions. Once I made my backup copy, I realized that I might run into a problem with the floppy drive – what if it subtly ruined my disk? Then one floppy would apparently be bad, so I’d feed the next one in, and the evil drive would chew that one up too. Hmm – better have three copies. I decided to keep one in my lab dsek, one at home, and one in my car. But then I started thinking of the unlikely – but still possible! – combinations of drive failures, fires, accidents, etc. that could still wipe me out. In the end, I had, I think, five separate copies of the dissertation in progress: one back at my apartment, one in the car, one in the lab desk, one back in a drawer by my hood, and one in my coat. I never needed any of the backups at all.

But it was a comfort to know that they were there, and mentally I needed all the backup capacity I could get in those days. Late one night I was awakened by a host of fire trucks roaring down the street. I lived only a quarter-mile from the chemistry building, and I found myself wondering, there at three in the morning, if that’s where they were headed. Ah, but I had my latest dissertation disks. But. . .I also had all the hard copies of my NMRs there in my lab. Aargh. (I should note that digital backups of NMR data were quite rare back in that era, at least in much of academia). What if the building caught on fire?

Worse, what if I’d been the cause? Had I really turned off that heating mantle when I left at midnight? Or did I just think that I had? Wasn’t there a bottle of hexane in my hood? (I did mention that this was three in the morning, right? Why the brain gets into these loops at that hour is a mystery, because that kind of thinking is normally alien to me, as my wife, to whom it’s second nature, will tell you). So I sat there, wondering if my lab and my data were at that moment going up in flames, until I finally rolled out of bed and called the lab. Ring. Ring. Ring. “Hello?” I recognized the voice – it was Randy, down the hall – but I suddenly realized that I didn’t know what to say to him. “So the lab’s not on fire?” didn’t seem like a good conversation starter, so I just hung up, and went back to sleep.

The next day I made my late-morning entrance into the lab, and ran into Randy. “How late were you here last night?” I asked him. “Oh, really late”, he said, and looked at me. “How did you know?”, he asked, and I looked embarrassed. “Hold it,” he said, “that was you, wasn’t it? You must have heard all those fire trucks going past! Thought the lab was on fire, didn’t you?” All I could do was turn red, because he had me.

Comments (29) + TrackBacks (0) | Category: Graduate School


COMMENTS

1. tom bartlett on November 20, 2007 9:22 AM writes...

"Time machine" will help some, but I think in the future, we'll all be backing up off-site to Google Servers sitting under a mountain in Colorado or somewhere.

Permalink to Comment

2. Bunsen Honeydew on November 20, 2007 9:49 AM writes...

I wrote up in 2003 and had 4 rewritable CDs with all my data on them (so like you, five copies if you include the hard drive). I would erase and rewrite them one at a time. Unless the CDs were with me they were never all in the same place. I too was paranoid of a fire so even if I was just going to the store for a coke I would bring my CDs with me.
I also would save my work as a new file every hour or two; I have lots of file like "Ch 4 Nov 20 1am". I was paranoid the file would be corrupted if I didn't play it safe. I think I wound up with about 200 files by the time I was done. I also got into the habit of saving my work every time I stopped to think. Finish typing sentence...apple+s...write another paragraph...apple+s

Permalink to Comment

3. Nick on November 20, 2007 9:50 AM writes...

When I was writing my dissertation (in somewhat more recent times), I had relatively recent copies of it on two seperate PCs, a copy on the departmental drive, and copies on two Zip disks, one of which was left in my apartment and the other of which lived in my briefcase. (I swapped the two Zips every couple of days to keep the apartment copy recent.) I was not particularly atypical among my peers.

Even now, I have the photos my wife and I have taken over the last few years on two internal hard drives, an external hard drive, and a DVD that is never more than a few months out of date.

Permalink to Comment

4. Nick on November 20, 2007 9:54 AM writes...

Oh, and as Dr. Honeydew has reminded me, I kept a rotating collection of at least the last five "archival" backups saved to separate files, in case Word corrupted one of the files beyond retreival. (After seeing Word destroy a paper that a postdoc colleague of mine had spend a week writing, I didn't consider this fear to be even slightly irrational.)

Permalink to Comment

5. eugene on November 20, 2007 9:55 AM writes...

I already back up my stuff onto the Google Server (and microsoft and yahoo). The only thing I find I need physical memory for (key drives or burned CDs) is the copious amount of NMR data which takes up way too much room. Why does one file from Bruker take up so much space?

But all my papers in progress, and the joke that is the thesis at the moment, are only backed up as emails to three different addresses at the end of the day, and all the previous backups are deleted. In case of fire that destroys the computer and the backup CDs at the lab desk, I'm hoping that a description of the NMRs (experimental format) will be enough. Considering I have easily >50 new compounds, and probably a lot more right now, all the printouts of the spectra will probably take up a lot of room.

The data that I'm absolutely not worried about are the published papers, because in the worst case scenario, I can just go to the ACS website and copy and paste (and make a lot of changes in format).

Permalink to Comment

6. Lucas on November 20, 2007 10:22 AM writes...

When I was writing my undergrad thesis I kept a couple of copies, on my computer and on verious servers. Good thing, too - I think during the process every server housing one of my copies went down at least once, and late in the game my backlight went out and I had to send my machine back to Apple for repair. None of the copies were actually lost, but having duplicates made it possible to keep working.

I used to do the hourly "in progress" backups too, in case Word ate one. But by the time I was writing my thesis I had started doing all of my writing in LaTeX, so I had pretty much stopped. BBEdit just doesn't eat files the way Word does. The only time I saved multiple versions was when I was trying to do some weird formatting markup, so I'd have a pristine copy in case I made it irreparably ugly (difficult in LaTeX, but not impossible).

Permalink to Comment

7. Tom on November 20, 2007 10:33 AM writes...

I have had the exact thing happen to me.... many times. Fire trucks racing to campus, I had just left at 2:00 am hmm, did I leave something on??

Permalink to Comment

8. GA on November 20, 2007 10:44 AM writes...

So this brings back memories of a *very* cold winter evening in the midwest. I had just walked back from the lab having set up a Raney-Ni reflux, and I remember wondering if I had set up the condenser properly in the hood. I debated long and hard whether to walk back in the -40 windchill to make sure that all was well. Finally I decided not to, although I was convinced that I would walk in to an army of fire-trucks surrounding the Chemistry building the next morning. No fire-trucks, though - it was fine.

Permalink to Comment

9. DSKS on November 20, 2007 10:48 AM writes...

I'm a seasoned backup 'noid myself, although fortunately I'm a bit more organised about it than I was when I was writing my thesis. I'd often get mixed up as to which version was the latest, and have to go trawling through all the file prefs to check the dates. But even that tedious process assumed that the last version was on the computer I was sitting in front of, rather than one of the other four I used for writing and backing up.

So, although I was fairly sure my work was safe - barring a global apocalypse - I was just never absolutely sure of where it was.

Y'live y'learn.

Permalink to Comment

10. Giagan on November 20, 2007 11:36 AM writes...

Seriously? When I was writing up my thesis 3 years ago, I had the latest version on the group's nicest laptop Mac. Other than that, I had a zip disk I would use for a backup every couple of days.

It's surprising because I am typically an obsessive person.

Permalink to Comment

11. TG on November 20, 2007 11:53 AM writes...

The fire trucks head to my campus at least 4-5 times a week because some moron in the dorms sets them off. Unfortunately I live on the road that these fire trucks head down each time to get to campus...

I can't even say how many times I have gotten home after setting up something like a LAH reduction only to hear the fire engines roar by. Now I am to the point where I just think to myself: Fuck it, if it burns down it burns down.

Permalink to Comment

12. Sarah on November 20, 2007 1:39 PM writes...

I was writing up in 2005. My laptop hard drive died the week I had decided I would finally start writing my dissertation. Luckily, I had recently installed Service Pack 2, and since I had read about all sorts of problems with it, I had backed up practically everything the month before. The only stuff I lost was music and some manipulated data (but I still had the raw data on disk).

Once I started writing, I backed up obsessively. Every night before I left lab (usually 2-3 am), I saved a copy to my jump drive, the network drive, my hard drive, and emailed a copy to myself.

Permalink to Comment

13. Brooks Moses on November 20, 2007 1:50 PM writes...

A few years ago, some of the people I was working with converted me to the "version control system" sect of the backup religion. I ended up using the TortioseSVN front-end to Subversion, running originally on a lab computer, and then to a server I was sharing with some friends off-site.

This has, in my experience, a number of advantages. One big one is convenience -- it's 30 seconds to backup my work, and I got in the habit of sending the changes upstream every half-hour or so of significant work, because it's so easy to do.

Another advantage, with my working style, is that it meant that I tested my backups. I had working copies on my home computer, on a laptop, and on my office computer, and all of those got synchronized from the source-control database whenever I started using a different computer. If something had gone wrong with the database, I'd have known quite quickly -- rather than finding out the backup was broken only when I desperately needed it not to be.

Oh, and it's pretty easy to make a single-file backup of the whole database, to keep that from being a single point of failure too.

So, yeah. At one time, I think I counted that, in order to make my thesis disappear, nine different hard drives in six different computers would have to simultaneously die. And that wasn't counting the copies in the University's file server and its professional-grade backup system, nor the weekly tape backups of another server a copy was on.

What, me paranoid?

Permalink to Comment

14. HelicalZz on November 20, 2007 3:21 PM writes...

I had only one experience with hard disk failure. I cost about a weeks saved work, but it wasn't too painful.

Today, I save everything to the company network by default, which is professionally backed up regularly. There is nothing on my desk PC but the software and maybe a couple of current documents (and shortcuts to the network). Been doing this for years. At home, I back up every couple of months except for things like Quicken which are more frequent.

Permalink to Comment

15. Jim Hu on November 20, 2007 4:20 PM writes...

A friend and I were working late on a very cold night over Xmas break in the lab in Madison when the building fire alarm went off. We had to decide whether to stay inside and burn or go out and freeze.

We compromised by going down to the lobby and waiting for the fire dept to arrive. Turns out that the distilled water still had frozen, ruptured a pipe, and the ensuing flooding had shorted out the fire alarms.

Permalink to Comment

16. psi*psi on November 20, 2007 5:31 PM writes...

My reliable flash drive of 3-4 years or so died on me last month. I almost lost three years' worth of data! Fortunately, the problem was only that the little USB thingie snapped off. I was able to enlist some nimble soldering assistance and managed to recover everything.

Permalink to Comment

17. Dennis on November 20, 2007 5:47 PM writes...

I have no backing up game.

I was working on a presentation last year and I always save my work on either my hard drive or flash disk and then would paste it into the other drive and replace the existing copy. Unfortunately I got distracted and ended up replacing the version with all the work in it with the version that just had the title and closed powerpoint. I only lost a couple of hours of work, but that sure made me feel dumb.

Permalink to Comment

18. Wavefunction on November 20, 2007 5:49 PM writes...

Hmmm...this is interesting. My comments got flagged probably because of the capital letters and the link. Let me try saying this again: Stem cell breakthrough! See the New York Times.

Permalink to Comment

19. milkshake on November 20, 2007 6:46 PM writes...

I have been ignoring warnings "imminent disk failure" on a PC that controlls one of our HPLC. Probably full disk I thought - so I deleted the old stuff to free up the space and it behaved fine, for a while. "I won't need any data and we have a disk image so if something happens we won't need to re-install the software piece-by-piece.

Then the disk quit I found out that getting a compatible hard drive for this antiquaed PC was not easy - it took two weeks (and the online store forgot about the order, then they re-send it by ground) Then it turned out that all the favorite HPLC methods and report layouts we use were not on the image... I cursed myself many times.

Permalink to Comment

20. bcpmoon on November 21, 2007 3:01 AM writes...

I wrote my thesis in 2000 and made two back-ups every day on diskettes. Of course, as you never crash when you have back-ups, right? Then, on the evening before I had to print, copy and submit, the hard disk had the one and only bad sector in its entire lifetime, right in the middle of the file. I could recover the thesis, only some formatting in the experimental part was lost. I think that was a warning by the data-gods...
And a fire-truck story: On evening in the lab I went to get some dry ice. On the way I noticed some curious draft and some minutes later some curious red trucks. When I returned to the lab, I found that a LAH-reaction in the neighbouring lab had dismantled a hood, destroyed the door to our lab and terrified my co-workers. Fortunately, the guy himself was on the loo. Interestingly, the shock-wave passed through the lab without crashing any of the many solvent bottles off the shelves.

Permalink to Comment

21. AHD on November 21, 2007 7:06 AM writes...

Since I'm writing in TeX, I keep all my thesis directories under version control as a Subversion repository. Currently I have three checked out copies in two different cities.

Regional thermonuclear war, anybody?

Permalink to Comment

22. Jeff on November 21, 2007 8:37 AM writes...

While I was writing up my PhD thesis I had electronic copies on three different always on and always up mainframes (gmail schoolwebserver and my own personal computer) and would run a chron job to synchronize them if necessary. I even had burned CD's of backups made every night automatically on a cd-rw. Yes paranoid I know, but I knew I wouldn't ever have to start over from scratch.

Permalink to Comment

23. Chris on November 21, 2007 4:43 PM writes...

Backups very important when in grad school!

When doing my MSc, I used a program called Foldershare to automatically synchronize documents on my work PC, my home PC _AND_ my laptop. All thesis documents (original data, analyzed data, papers, references, thesis, etc.) would be automatically synchronized between the three PCs. I also did a weekly backup to my very-much off-site FTP as well. The odds of all four data sources crapping out on the same day are very remote. The beauty of this system is that it's all automatic and Foldershare will synchronize files within minutes of them being changed/saved.

Alas, foldershare was bought by Microsoft...(still works and is still free, for now!)

For my PhD, I'm also scanning my lab book to PDF format on a weekly basis. So being a good grad student, I can do data analysis on a Friday or Saturday night on my laptop and not have to haul my big-freaking lab book with me. I can just open the PDF instead. In case the lab the lab ever goes up in flames, I won't "lose" my lab book.

But I'm an analytical chemist, so I don't have to worry about these weird reactions going up in flames and blowing up adjacent labs...jeez!

Permalink to Comment

24. Renan "Renan_S2" Birck on November 22, 2007 9:53 AM writes...

I have always read your blog, but never commented, mainly because I'm not into chemistry/pharmacy research.

But I can say that floppy disks really SUCK. When I had to use them, I used to do at least 3 or 4 backups.

Now I usually backup important things to at least 2 of those: CD-R/DVD-R, USB drive, memory card on cellphone, and e-mail.

Permalink to Comment

25. Ben Hemmens on November 23, 2007 3:27 AM writes...

I think the ability to think back and correctly recall what one did / didn't do is essential for most working scientists. If we are honest, what to do next with today's baffling results is often a matter of a hunch on some detail that we could not have imagined being relevant yesterday and therefore never got anywhere near a mention in our lab notes.

And the way we develop that ability is, we all get up a couple of dozen times in the middle of the night and head back to the lab to check whether we really left enough buffer to get the hplc through the night, really turned the pump down/ the light off, etc. After a while we learn to decide this stuff while still lying in our bed. And of course our fear of our boss / the guy we borrowed the column from, etc. (if we guess wrong) also gets ground down gradually. The thought "the hell with it, if I get sacked tomorrow, I get sacked tomorrow" is, I suggest, a crucial step in the development of our own embryonic scientific personality.

Permalink to Comment

26. Meyer on November 24, 2007 12:17 PM writes...

For all those situations you've here described, I'd definitively use BOS (Backup prOxy Server) software - it is free for students and home users, and the best data insurance I've ever had.

Permalink to Comment

27. Ken Mitchell on November 24, 2007 9:45 PM writes...

1. Robert Bruce Thompson, a writer of technical and computer books, (www.ttgnet.com) does daily backups to DVD-RW and stores them in a CD wallet which goes with him ANY time he leaves the house. I think he takes it with him when he walks the dogs.

2. A 500GB NAS (Network Attached Storage) drive costs less than $200 these days, with backup software included. Plug it into your network switch, and let it work.

Permalink to Comment

28. David on November 28, 2007 2:04 PM writes...

My computer died in the middle of writing my dissertation. Fortunately I had everything backed up.

Permalink to Comment

29. IT Guy on August 28, 2008 8:52 PM writes...

Here is some information that can help. This is standard practice in any IT shop:

1. Use a revision control system. This saves EVERY change you ever make to a document. Subversion is the best free open source tool out there for this.

http://subversion.tigris.org/

Don't be scared off by the fact that it is for developers. It works for pretty much anything.

2. Have a back-up and recovery procedure, both manual and automated.

Once you check-in all your changes to Subversion, schedule a backup for the ENTIRE REPOSITORY to an external device. And know how to recover it.

*** You can save documents all over the place, but at the end of the day a revision control system like Subversion will save your ass every time ***

Permalink to Comment

POST A COMMENT




Remember Me?



EMAIL THIS ENTRY TO A FRIEND

Email this entry to:

Your email address:

Message (optional):




RELATED ENTRIES
XKCD on Protein Folding
The 2014 Chemistry Nobel: Beating the Diffraction Limit
German Pharma, Or What's Left of It
Sunesis Fails with Vosaroxin
A New Way to Estimate a Compound's Chances?
Meinwald Honored
Molecular Biology Turns Into Chemistry
Speaking at Northeastern