A Tale of Woe
Backing up your photos–properly archiving everything–is something we all say we’re going to do. But how many of us consistently do it? My complacency on this issue recently turned into the nightmare of all nightmares, and I’m here to tell you about it: kids, don’t try this at home!
I changed out laptops about 4 months ago, trading in my old MacBook Pro for a new one. Along with all of my work-related files for Nola.com and The Times-Picayune, the old machine contained a monster series of nested folders of my landscape photography, all of which I regularly backed up to an external hard drive using the Mac’s built-in Time Machine application. All good so far, right?
Well, when getting ready to switch to the new machine, I took that big set of landscape photography folders and did a manual additional copy over to an external disk–you can’t be too careful, right? It was around 40 GB of material and I left it to copy overnight. The next day I did some additional cleanup of the machine, getting ready to turn it in.
Fast forward two months, and I needed to print images for the upcoming ‘Fall for Art’ show at the Henry Hood Gallery in Covington, Louisiana. I was looking forward to this process, thinking about some specific images from the Atlantic shore that I wanted to trying printing for the first time. I hooked up my external hard drive and went to the set of nested folders. Clicking the first one open I found….nothing! A blank, empty folder! I quickly tried several more—and found them all to be empty. Somehow, my folder structure had copied overly perfectly, but none of the files! I desperately tried everything I could think of, even pulling out every other external hard drive that I owned, looking for these photographs—none of it was there.
I didn’t know if my overnight copy timed out somehow or if some other gremlin was at work, all I knew after a careful, methodical search of every possibility was that the last year and a half of my landscape work–all of it—was GONE. To make matters infinitely worse, I had replaced my old Time Machine configuration with the new one that I’m now using on the new laptop–effectively throwing away the second, last copy of these images.
A Glimmer of Hope
Despair, depression, desperation: all good words to describe that day. At the end of it all, I held onto one slim, glimmer of a possibility: what if my old machine was still around, and even more unlikely, what if it had not been wiped off and reformatted? I sent an email to our I.T. department and discovered that yes!–my old laptop was sitting on a shelf, untouched since I surrendered it. And yes, out of sympathy for me (and the fact that my friends in I.T. know I’m never been a problem child for them)–they said I could borrow it for awhile.
I knew that the recovery process was fraught with difficulties and not at all a sure bet, but certainly worth the effort. I did a bunch of online research, tried a couple of recovery programs that turned out to be duds before settling on a wonderful, reasonably-priced solution from the fine folks at Wondershare: Wondershare Data Recovery. I’ve used some of their other products and have always been pleased so giving this one a go was not a hard decision.
To use the Wondershare Data Recovery application, you need the machine you hope to recover files from hooked up to another machine, upon which you have enough free space and will run the program from. Because my old machine was a 4+ year-old Mac and my new machine only a few months old, I opted for a Firewire 800 cable to connect the two. (If you were going to run a recovery between two newer machines, you’d want to do this with a Thunderbolt-to-Thunderbolt data cable since your transfer speeds will be quicker.)
The process is simple: Have your new machine running and the cable hooked up between the two boxes. Then start the older machine, while holding your finger down on the “T” key. The old machine will boot up as an external disk, viewable on the desktop of your new machine. You’ll see a symbol that I think stands for ‘Firewire’–I think it looks a bit like a symbol for something that’s radioactive–and that will dance slowly around the screen of your old machine while mounted in this way.
Once they are hooked together, launch Wondershare Data Recovery. The main menu will appear with 5 options. I suggest you try the first option–‘Lost File Recovery’–first. That didn’t work for me, so I went to ‘Raw Recovery’ which began the process of creating a new File Allocation Table for all of my deleted material. It finds everything that’s been deleted–Word docs, Pdfs, jpegs, tifs, pngs, various video file formats–and displays a list for you to peruse. In my case, the program located 75,000 jpegs images plus a slew of other material. Once you hit the ‘Recover’ button at the top of the window, get your credit card out because this is when you’ll have to pay for the privilege of using their product.
As of this writing, the price for the fully-functioning Mac version of the program was $89.95 U.S., a bargain by anyone’s measure.
The Nitty Gritty
What I’ve found is that the program is very good at finding most of my original files–the exposures as they were ingested off the compact flash cards to the computer. It’s not so good at locating really large, fully-toned images. I’m not sure what the exact file size limit it, but there definitely is one. Lots of toned images in the 20-25 MB size were located and recovered intact; bigger files, not so much. But at least with the originals to work from, I can go back and retone the images, immensely preferable to having nothing at all! Also, although it found and generated preview icons for layered Photoshop (.psd) files, it was generally unable to open these, so if this is your main recovery need, this program may not work for you.
So now I’m in the sorting process: all 75,000 jpegs have to be looked at in some fashion. This is actually very interesting: I never realized it before but every time you upload an image or work on it in Photoshop, different sizes get created and saved. You’ll have tiny little thumbnails still there, getting recovered, and other sizes as well. What I ended up doing was taking a batch of the files–the jpegs get saved out in batches of 10,000– doing a sort-by-size for the batch, which the program allows you to do–and throwing away everything smaller than an arbitrary size of 200K. (I figured that something saved as a jpeg of 200K might be useable, anything smaller, probably not.) This would eliminate 6-7000 of the files right away, leaving me with a more manageable 3000 or so to look through.
Even then you’ll find lots of images saved on your computer that you’ve never seen before! Things like small files that are part of a program you might have on your machine and never use, for instance.
I’m doing the sort process in Photo Mechanic, which not only allows me to browse everything, but will let me create and copy files out to new folders on the fly. So I’m slowly recreating all of my main folders, many times combining photo trips by location. When I’m finished I should have most of my old images back, at least in their original, raw state.
So what’s the takeaway? Obviously, that you can’t have too many back ups. Back up early and often is the dumb-as-rocks lesson here for me. I’m experimenting with a few things that I’ll mention as well.
I took about 2GB worth of toned images that I did NOT lose–the print order folder that I had moved over to the new laptop separately–and copied that monster up to my Google Drive home. This took about an hour but did create an additional, safe location for these files. The downside is that Google Drive does not seem to generate preview icons, so you’re left with available files buy now thumbnails.
I also have a Flickr Pro account which allows 1 TB of storage for, right now, a very cheap $25 U.S. per year. (Note: Flickr Pro accounts are in flux as Flickr messes around with their products. Stay tuned.) This does generate thumbnails and will allow big monster copies up to it. I did this with a bunch of material last night and found it worked very well, although this also is a slow, cumbersome process. The main irritant for me with the Flickr account is that the default settings, which I can’t seem to change, start with everything you copy up being publicly viewable and searchable by the world; I want this stuff to be my private archive. This is possible but requires checking off a bunch of boxes on each batch you upload, a pain in the ass, I think.
So my plan at this point is:
1. To use the Mac’s Time Machine utility to do regular backups of my computer;
2. To copy all landscape work out to an additional external hard drive;
3. To upload all finished personal landscape work to my Flickr account;
4. To make DVD copies of the raw files from every shooting trip and store them in a different location.
Have I learned my lesson? I certainly hope so. Only time will tell!
So what’s your photo archiving process? Do you have one? Do you use it regularly? How did you turn it into a habit that was easy to follow? I’d love to hear how others are handling this ever-so-crucial chore. Please let me know!
Hi, I’m Andrew Boyd, a.k.a. The Discerning Photographer, and I hope this post has been interesting and informative. Please leave me a comment about it, let me know what you’d like to see more of on the site! You can also sign up for email delivery of all future articles or my RSS feed. Or subscribe to our Facebook page ,Google+ pageor our Twitter feed. Thanks!–DiscerningPhotog