Posted by Jim DeWitt on February 19, 2008 at 2:14 PM
Last night I had a chance to hear Nat. Geo. photographer Joel Sartore speak. He was very good; if you have a chance to hear him, you should take it. He's a passionate guy.
In response to a question from the audience, he talked about the difficulties of archiving digital photographs. The Mean Time Between Failures (MTBF) for most consumer-level hard drives is usually 100,000 hours, just 11.5 years. No one really knows how long CD's or DVD's will last, but we all know the effects of scratches to their surfaces. Changes in software can also make perfectly preserved images unreadable, as can changes in media. Does anyone besides me have data orphaned on 5.25 inch floppy disks, or 3.5 inch disks? Film and prints, by contrast, have a life measured in decades, longer if high quality stock is used. Sartore is concerned that that the practical consequence is that only a few dozen of the very best photos from each photographer will end up being preserved.
All of which leads me to my question: what are the backup and archiving strategies that NPN members are using?
We mix it up. We have backup on removable drives, CD and DVD. Our cirucmstances emphasize an often overlooked component of backup- Safe storage of the media. The most archival media in the world isn't going to do you one bit of good if it's lost in fire, flood, tsunami, earthquake or theft. We keep mulitiple backups in multiple locations. Really, really hard to do that with film. And as soon as I get around to scanning the last of our film archives, they're likely to go into "safe storage" far from our current techtonically active location. And of course they'll be completely out of reach for use, so the "backup" I make will in fact become my prime source.
An often-ballyhooed disadvantage of digital media is the question of archaic reading technology. In our experience that's all hoo with no bally. With each new generation of reader and media, it gets easier, faster and cheaper to generate multiple and replacement backups. Drives are so cheap now that I'm in the process of transferring files from our backup CD/DVDs onto them. By the time CD/DVDs are no longer supported, I'll have made several generations of backups onto newer media. Goodby and good riddance to the originals.
Posted by David Bostedo on February 19, 2008 at 2:45 PM
I agree with Hank on the "all hoo, no bally" comment (nice phrase, too, Hank). If you have old floppies Jim, you can likely find a disk drive that is USB or Firewire, and simply transfer the contents. (I have a USB 3.5 inch drive for that purpose. A 5.25 might be harder to find - although maybe a COM port one would be available somewhere.) And since you can get over 1300 floppies on a single 2 GB USB key, finding some other way to store them should be cake.
Posted by Stephen Fogle on February 19, 2008 at 3:08 PM
I've been using gold standard CD's & DVD's for archiving digital files. The media is made by Ma-ma company and contain a 24 karat gold reflective layer that, because it does not oxidize if the atmosphere reaches it, can last from 100 to 300 years in storage. Normal media has either a silver or aluminum alloy being used as the reflective surface so when you get a scratch or a worn spot in the poly casing around it the oxygen will eventually create a pinhole in the metal making them unusable (music CD's and DVD videos will skip under the same condition). The gold CD's and DVD's cost more than the standard fare (I buy them in bulk online) but for important things like our photographs, it's worth the extra money.
Posted by Jesse Speer on February 19, 2008 at 7:06 PM
I have added Jungle Disk and Amazon.com's S3 storage solution to my backup/archive strategy. It's scalable, affordable, remote, backed by a big name, etc. I think it's the ultimate remote storage solution, to compliment my local storage setup. In fact, I've dropped CDs and DVDs in favor of this method ... which is currently my 3rd line of defense.
Of course, I don't expect this service to be around in 100 years, but something else is sure to take its place. Remote/online based storage is getting more robust, more affordable and faster ... and will be around for a long while. It's perfect for a backup set.
I'm also starting to make a lot of prints ... in case the sun decides to go nuts and causes some major worldwide electromagnetic disturbance that wipes out everything digital.
Posted by Jim DeWitt on February 21, 2008 at 3:28 PM
I'd hoped to get more feedback on this. The problem breaks down into three parts: the physical preservation of the data, the machinery to read it, and the software to interpret it. All complicated by the sheer volume of data.
There have been good suggestions for the first part: physical preservation. Gold-based DVD's probably lead the pack. We don't know how long they will last, but it's longer than anything else we have got. So we have the techniques to preserve the bits - the 1s and 0s - for a long time. Certainly longer than the life of a third party vendor in an industry as volatile as computers. There are capacity issues.
The machinery is more problematic. The design Mean Time Between Failures is so short that the chances of existing equipment surviving for, say, 100 years, is nearly zero. So we're talking replicating existing equipment at some point in the future.
The software is still more problematic. Both the software to actually read the data from the equipment and the software to interpret what is read. Think about all the dead computer manufacturers. I work for a company that used to use Wang computers company-wide. All that data is completely orphaned. Anyone here invest in Toshiba HD-DVD?
A few have suggested we migrate our data to new media and new formats as those new media and formats develop. Sure, except that the sheer amount of data makes this dubious. I shot more than 50 GB of photos in two weeks in Florida. Others on this forum undoubtedly create much more than that. We're talking about migrating terabytes (petabytes? exabytes?) of information for each of us. Cameras keep capturing more information; it's only going to get worse.
Now none of us is Ansel Adams, but some folks posting here should have their photos preserved after their lifetimes. And NPN, as big and successful as it is, represents only a tiny fraction of serious nature photographers, themselves a tiny fraction of total digital photographers. Who's going to keep migrating all that data to a then-readable form?
A side note: when I was a kid in remote Alaska, my folks made Kodak 8mm movies of my brother and I. The film stock is now to fragile to run through an 8mm projector. It is prohibitively expensive to have a pro with museum archive equipment scan the movies frame by frame. So those films are, for all practical purposes, lost forever.
The only time I've encountered any software problems is with files saved to Genuine Fractals proprietary format. No one else reads it (or did last time I checked), so if you don't keep up with their software, you're screwed. As seen by the recent HD format war, manufacturers can abandon reader technology, so there's no "guarantee" that within the data storage life of your discs, they won't become obsolete even if the data is still good. Might be a problem for your anscestors, might be a problem for you sooner than that. But rest assured that probably within the next 20 years CD/DVD media as we know it are toast. No matter how you strategize it, you (or your anscestors) are going to be updating formats.
The biggest downside I see for digital capture and storage is both its blessing and its curse. We can capture so much easily and cheaply that we do lots. But it's so easy to kull, we also do that. Film's greatest archival characteristic is the tendency of people to hang onto it longer, at least till it becomes historically or artistically signicant.
I have a couple of friends who knew and worked with Ansel Adams and still have access to his archives. They agree that in spite of his great rep and work, there's a mountain of schlock in his archives along with the good stuff. The only reason it's in there is because he never got around to categorizing it, much less throwing it away. We're neat freeks and space freeks and quick with the delete button because we can. Not so with most of the classic film photographers. Much of their "schlock" from the days of capture now has historical or artistic significance that didn't exist at the time of capture. The only reason it survives today is because the photographer was sloppy about dumping it.
Interesting enough, both of these photographers believe that the biggest loss of historically or artistically significant digital data will be to the delete button and not to media deterioration or obsolescence. Their point is that we're careful about backing up and migrating our favorites, but eventually let all the other stuff slide. But in fact, we're probably not good judges of what's going to be important 50 or 100 years from now. I take their point and keep migrating the whole mass to bigger, faster and cheaper storage while not being quite so quick with the delete button or round file. It will be up to my heirs to decide what, if anything in the mess, is worthwhile. Too late for me to worry about it by then.
Posted by Steve Foss on February 21, 2008 at 4:10 PM
Great discussion, everyone. The more you talk, the more I learn.
So keep talking!
Posted by David Bostedo on February 21, 2008 at 4:32 PM
Jim - It sounds like your looking for solutions that don't exist yet. The suggestion given, and that you made, are what we've got right now. I will say, though, that you talk about massive amounts of storage needed - but I'd think that if things keep going the way they have been, a few terabytes won't seem like so much. (I realize there is a limit to current technology, but forecasting what's down the road is notoriously difficult.)
And as far as devices still being made to read the media in the future, if the amount of data and users warrant it (meaning make it profitable) someone will continue to make, for instance, DVD readers, or computers capable of reading hard drives. Keep in mind that when you were using Wang computers, they were far, far, far from ubiquitous. Same goes for HD-DVD. In the case of DVD, current hard drive technology, etc, they are pretty much ubiquitous.
So I think that overall, the current accepted technologies will be around for at least a little while, and if/when they aren't, transferring probably won't be prohibitive. (A lot of people have transferred thousands of slides to digital, for instance.) Will things get lost/left etc? Sure, but no more than did with any other recording/archiving media in history. So of course, what will be true for most of us, as it has been the entire history of photography, is that our photos will not be maintained after our deaths (or that of our families) and will be "lost". I don't know that this is such a big deal.
Here's a couple of interesting points, up to you to judge whether they're additional insights or distractions.
1. We've got a mass of photos in the portrait/commercial/industrial side of our business. What happens to them when we sell the business? Do they go with us? And what does the copyright read if we keep them? In fact the business carries our name, so when we sell the business, are we also selling our name? The short answer is we need to change the name of our business (along with the signage and forms, etc) before we sell, but do so soon enough that the new name can become established and acrue recognition for value in the sale.
2. We're doing the inheritance/will/surivor stuff that everyone is supposed to do. Our attorney asked the question "What about the photos?" If we REALLY want them to continue on into future generations, we need to make that provision in our will, and probably set up some kind of financial trust to fund their maintenance. Without such a setup, our heirs could easily take one look at the mass, back a dumpster up to the door, and get rid of all of them along with the "treasure" we've accumulated living in the same house for 35 years. How are they to know what's good stuff and what's schlock?
What is the name of your business?
Do you have a will, and does it specify, much less fund, management and disposition of your photos?
In all likelihood the answers to those questions will determine more about the longevitiy of your photos than any consideration of archival live of film and the fate of digital media.
Posted by Dave Gill on February 23, 2008 at 3:28 PM
I've just added a 2 terabyte NAS to my network. It has four 500 gig drives and is configured in a RAID 5 so if any one of the drives goes bad I still won't lose any data so long as I replace that drive before another drive goes bad. It isn't a perfect solution, but it should make a good addition to backing up on DVDs and off-site storage.
Posted by Kent Staubus on February 24, 2008 at 10:55 AM
About ten years ago my mother was cleaning out my grandmother's things and found a shoebox full of negatives from a Kodak Brownie. They were family photos from the late 20s to the early 40s. I was able to scan them and make great prints. I have photos of my dad from the mid 1920s, and photos of grandparents from the 1890s. Archivability is one of the biggest things for me because I intend my photos to be a record of the Northern Plains, and my intended audience hasn't been born yet.
My brother-n-law is a systems consultant for the military, and archiving data is a huge part of his job. I have a commercial photographer friend who works as an archivist for the Smithsonian. I also work as a sales rep for a brokerage company that handles Kodak and have talked to some of their research people in Rochester. I asked them what their advice was. Here's what I now do. Not one of these people really trust ANY digital storage. For really important photos I make an archival 8x10 print. For many others I make an archival 4x6 print. Two copies each. For the rest I make one gold Delkin CD and a gold Delkin DVD. I do not trust any hard drives at all, so I skip that for anything but photos I want handy during the next year. Believe it or not, I still shoot some silver black & white 4x5 of family shots because I know those are archival for at least 120 years.
Now you're talking Kent. I particularly like the archival print addition to the strategy. The biggest reason for me is that great big blank space on the back! You can write in all sorts of ID info, so there's no confusion for future generations.
I just finished a contract scanning and printing a bunch of heirloom pprints, some dating back to the early 1900's. Neat project, but it's been kinda sad to watch the family debating exact ID (plus date and location) of many individuals in the photos. The family members old enough to remember often can't, plus they are strung all over the country. The prints I made are being circulated now in an attempt to glean as much info as possible, but it's kinda discouraging for the owners.
Posted by Scott Buckel on February 26, 2008 at 1:12 AM
For me there is a more scary question - you can save files in a number of ways with many kinds of media and likely bridge them with little to no problem. The question I have is how do you verify that the images through multiple reads and writes is exactly the same? There are starting to be a few tools out there, one of which is ImageVerifier.
A couple of years ago I had to pull some data generated in my lab that was saved on a validated network, with validated means and a couple of the important files were corrupted in minor ways (fortunately we were able to repair the files). I know that none of use invest the money that a pharmaceutical company has for its data. That whole process is begining to be a worry for me.
So how do people verify that their archive of images are truly safe?
Discussion
14 repliesIn response to a question from the audience, he talked about the difficulties of archiving digital photographs. The Mean Time Between Failures (MTBF) for most consumer-level hard drives is usually 100,000 hours, just 11.5 years. No one really knows how long CD's or DVD's will last, but we all know the effects of scratches to their surfaces. Changes in software can also make perfectly preserved images unreadable, as can changes in media. Does anyone besides me have data orphaned on 5.25 inch floppy disks, or 3.5 inch disks? Film and prints, by contrast, have a life measured in decades, longer if high quality stock is used. Sartore is concerned that that the practical consequence is that only a few dozen of the very best photos from each photographer will end up being preserved.
All of which leads me to my question: what are the backup and archiving strategies that NPN members are using?
An often-ballyhooed disadvantage of digital media is the question of archaic reading technology. In our experience that's all hoo with no bally. With each new generation of reader and media, it gets easier, faster and cheaper to generate multiple and replacement backups. Drives are so cheap now that I'm in the process of transferring files from our backup CD/DVDs onto them. By the time CD/DVDs are no longer supported, I'll have made several generations of backups onto newer media. Goodby and good riddance to the originals.
Of course, I don't expect this service to be around in 100 years, but something else is sure to take its place. Remote/online based storage is getting more robust, more affordable and faster ... and will be around for a long while. It's perfect for a backup set.
I'm also starting to make a lot of prints ... in case the sun decides to go nuts and causes some major worldwide electromagnetic disturbance that wipes out everything digital.
There have been good suggestions for the first part: physical preservation. Gold-based DVD's probably lead the pack. We don't know how long they will last, but it's longer than anything else we have got. So we have the techniques to preserve the bits - the 1s and 0s - for a long time. Certainly longer than the life of a third party vendor in an industry as volatile as computers. There are capacity issues.
The machinery is more problematic. The design Mean Time Between Failures is so short that the chances of existing equipment surviving for, say, 100 years, is nearly zero. So we're talking replicating existing equipment at some point in the future.
The software is still more problematic. Both the software to actually read the data from the equipment and the software to interpret what is read. Think about all the dead computer manufacturers. I work for a company that used to use Wang computers company-wide. All that data is completely orphaned. Anyone here invest in Toshiba HD-DVD?
A few have suggested we migrate our data to new media and new formats as those new media and formats develop. Sure, except that the sheer amount of data makes this dubious. I shot more than 50 GB of photos in two weeks in Florida. Others on this forum undoubtedly create much more than that. We're talking about migrating terabytes (petabytes? exabytes?) of information for each of us. Cameras keep capturing more information; it's only going to get worse.
Now none of us is Ansel Adams, but some folks posting here should have their photos preserved after their lifetimes. And NPN, as big and successful as it is, represents only a tiny fraction of serious nature photographers, themselves a tiny fraction of total digital photographers. Who's going to keep migrating all that data to a then-readable form?
A side note: when I was a kid in remote Alaska, my folks made Kodak 8mm movies of my brother and I. The film stock is now to fragile to run through an 8mm projector. It is prohibitively expensive to have a pro with museum archive equipment scan the movies frame by frame. So those films are, for all practical purposes, lost forever.
The biggest downside I see for digital capture and storage is both its blessing and its curse. We can capture so much easily and cheaply that we do lots. But it's so easy to kull, we also do that. Film's greatest archival characteristic is the tendency of people to hang onto it longer, at least till it becomes historically or artistically signicant.
I have a couple of friends who knew and worked with Ansel Adams and still have access to his archives. They agree that in spite of his great rep and work, there's a mountain of schlock in his archives along with the good stuff. The only reason it's in there is because he never got around to categorizing it, much less throwing it away. We're neat freeks and space freeks and quick with the delete button because we can. Not so with most of the classic film photographers. Much of their "schlock" from the days of capture now has historical or artistic significance that didn't exist at the time of capture. The only reason it survives today is because the photographer was sloppy about dumping it.
Interesting enough, both of these photographers believe that the biggest loss of historically or artistically significant digital data will be to the delete button and not to media deterioration or obsolescence. Their point is that we're careful about backing up and migrating our favorites, but eventually let all the other stuff slide. But in fact, we're probably not good judges of what's going to be important 50 or 100 years from now. I take their point and keep migrating the whole mass to bigger, faster and cheaper storage while not being quite so quick with the delete button or round file. It will be up to my heirs to decide what, if anything in the mess, is worthwhile. Too late for me to worry about it by then.
So keep talking!
And as far as devices still being made to read the media in the future, if the amount of data and users warrant it (meaning make it profitable) someone will continue to make, for instance, DVD readers, or computers capable of reading hard drives. Keep in mind that when you were using Wang computers, they were far, far, far from ubiquitous. Same goes for HD-DVD. In the case of DVD, current hard drive technology, etc, they are pretty much ubiquitous.
So I think that overall, the current accepted technologies will be around for at least a little while, and if/when they aren't, transferring probably won't be prohibitive. (A lot of people have transferred thousands of slides to digital, for instance.) Will things get lost/left etc? Sure, but no more than did with any other recording/archiving media in history. So of course, what will be true for most of us, as it has been the entire history of photography, is that our photos will not be maintained after our deaths (or that of our families) and will be "lost". I don't know that this is such a big deal.
1. We've got a mass of photos in the portrait/commercial/industrial side of our business. What happens to them when we sell the business? Do they go with us? And what does the copyright read if we keep them? In fact the business carries our name, so when we sell the business, are we also selling our name? The short answer is we need to change the name of our business (along with the signage and forms, etc) before we sell, but do so soon enough that the new name can become established and acrue recognition for value in the sale.
2. We're doing the inheritance/will/surivor stuff that everyone is supposed to do. Our attorney asked the question "What about the photos?" If we REALLY want them to continue on into future generations, we need to make that provision in our will, and probably set up some kind of financial trust to fund their maintenance. Without such a setup, our heirs could easily take one look at the mass, back a dumpster up to the door, and get rid of all of them along with the "treasure" we've accumulated living in the same house for 35 years. How are they to know what's good stuff and what's schlock?
What is the name of your business?
Do you have a will, and does it specify, much less fund, management and disposition of your photos?
In all likelihood the answers to those questions will determine more about the longevitiy of your photos than any consideration of archival live of film and the fate of digital media.
My brother-n-law is a systems consultant for the military, and archiving data is a huge part of his job. I have a commercial photographer friend who works as an archivist for the Smithsonian. I also work as a sales rep for a brokerage company that handles Kodak and have talked to some of their research people in Rochester. I asked them what their advice was. Here's what I now do. Not one of these people really trust ANY digital storage. For really important photos I make an archival 8x10 print. For many others I make an archival 4x6 print. Two copies each. For the rest I make one gold Delkin CD and a gold Delkin DVD. I do not trust any hard drives at all, so I skip that for anything but photos I want handy during the next year. Believe it or not, I still shoot some silver black & white 4x5 of family shots because I know those are archival for at least 120 years.
Kent in SD
I just finished a contract scanning and printing a bunch of heirloom pprints, some dating back to the early 1900's. Neat project, but it's been kinda sad to watch the family debating exact ID (plus date and location) of many individuals in the photos. The family members old enough to remember often can't, plus they are strung all over the country. The prints I made are being circulated now in an attempt to glean as much info as possible, but it's kinda discouraging for the owners.
A couple of years ago I had to pull some data generated in my lab that was saved on a validated network, with validated means and a couple of the important files were corrupted in minor ways (fortunately we were able to repair the files). I know that none of use invest the money that a pharmaceutical company has for its data. That whole process is begining to be a worry for me.
So how do people verify that their archive of images are truly safe?
Scott