I have read "The Dam Book" several times, listened to instructors discuss their file management process several times and still am not happy with what I am doing. Using the date of capture in the file name seems meaningless to me. Filing photos by date seems to me to be unimportant. Filing by subject can get confusing and complex. What system(s) for both naming and filing are you using and why?
Thanks lots for input!
If you're using a database system, then the only thing that really matters is that the file name is unique (I use simple serial numbers with a letter code to denote format). The database can do all the searching and cross-referencing for you so you don't have to search through files manually.
I started doing this quite a ways back when there weren't a lot of commercially available databases (the couple that were available were ridiculously priced) so I ended up writing my own.
I like the database method as well. You just have to remember to tag your images as you add them. Lightroom has a database type of interface as does PS Elements Organizer (windows version of Elements).
I'm pretty much following Peter's suggestions as outlined in The DAM Book. For my file naming, I'm using the following format: Barbee_x_WWYY_1234.dng (or whatever the extension happens to be)
where x is a camera code (just to keep from the possibility of duplication of file names and to quickly id which camera was used-in my case either the D100 or the D200), WWYY is a date code for week and year (instead of the exact date of capture--this saves two digits over using the full date), and 1234 represents the number from the original camera file name.
By the way, using date of capture in a file name is another way of minimizing the possibility of duplicating file names. Remember, your camera will count up to 9999 and then start over, so you could easily start duplicating file names. And if you're shooting with two cameras, it's even easier to duplicate names.
For me, this is easy because the file names are automatically generated when I download the images using Downloader Pro.
I used to keep all my files in a folder hierarchy, maybe something like this: Nature and Travel/Locations/National Parks/State/Olympic/rainforest. But this got way to cumbersome when it came to filing images. Many times, an image could easily be filed in two or more folders and I was wasting time in figuring out which one is should go into.
Now, my originals (DNG raw files and original jpg captures) go into "original" folders such as orig_001, orig_002, etc. Each folder is about the size of a dvd and all folders are backed up to two other hard drives (one goes off site) and two sets of dvds (one set off site)
Any derivitives (images processed for prints, etc.) go into folders marked drv_001, drv_002, etc.
the images in these folders are, because of the date code and file number, automatically arranged in ascending order, earliest to latest. This can sometimes be an advantage.
The trick to this is to apply metadata to the images as soon as you possible can. In Bridge, I add caption and location information and any keywords and ratings and labels (I'm not as good at applying keywords as I should be). Using templates in Bridge can make this go very fast.
After I've deleted the not-so-good images, I convert all the keepers to the dng format and then use a utility called Rank and File Lite to embed labels, ratings, and the filename into the metadata. I then move these keepers to their final folder (in this case the next available orig_xxx folder)
The images are then cataloged in iView Media Pro where I can search by just about any parameter I can think of. There's even a date finder that makes finding things by exact date shot.
In iView, I've also set up Catalog Sets that I can drop images into for sorting in other ways, like if I want to look at all my rainforest pictures at once or if I'm looking images to use for specific teaching programs.
By using a DAM application like iView or Portfolio, you can organize your images in any way you want and you're not stuck with any one way. I can view images by date shot, by location, by state, by ISO, by EXIV data, by Catalog Sets, or just about any criteria.
For my style of shooting, I find it interesting if not critical to know the date of a shot. It's cool to see how a given place might have changed over the years, or through the seasons. I also find it to be the simplest method of naming shots without accidentally duplicating a name. I keep a running file list with names and descriptions in Excel, which isn't the greatest, but it's software I already have and suits my simple needs.
I have also read Peter's book several times and I like his naming system, I use: buckel_yymmdd_camera file number just like Peter does. I like it since it generates unique file name. I copy the rated converted files into buckets just like he does. This whole process makes sense to me so it is easy to keep up with.
As i found that folders with specific information to be difficult to keep up with, much like Rod above mentions. I use Image Ingester to remane my files which makesthe rename mindless (that is a good thing - my mind wanders and gets creative on mundance tasks which makes files difficult if not impossible to find!)
I just moved 5 years of digital files over to this system about a month ago (having tried Peter's system for about 6 months). The move process sent very well and I think I can find the raw images faster and easier than ever before. Now I just need to learn to enter metadata more quickly!
I establish a folder for each species of life or unique landscape captured that day and load the images into them that way. So I may have a folder that reads: 5-15-07 male rose-breasted grosbeak. Within that folder, I select the images I like best and title them, leaving the extension in place. That folder then goes into another folder that reads: 2007 images. I have year folders from 2003 to 2007, and I really like having them arranged by date and species or location because I can use that as a recurring diary of bird migration and flowers opening to refer to over the years.
This is especially helpful when clients enquire which species they might expect during photo excursions, and because some want to target specific species. I have one couple coming early next month who are specifically interested in lupines, and by looking back in the last two years' folders, I can see that lupine blooms peaked on the first week of June both years, which helps me schedule the best day for the excursion.
Images that I process and tone stay in the dated folders, but copies get moved over to other folders labeled by species. I have a series of those. If I want to find a female wood duck, I go to the folder titled "birds" and into the "ducks" folder and from there to the "wood duck" folder, where my keeper wood duck images are. As I amass more keepers, I'll create more sub-folders, eventually creating "female" and "male" and "family" subfolders within the "wood duck" folder.
Most queries that come to me from the publications I sell to come as a species inquiry, such as: "I need an image of a male pine grosbeak on spruce branches," so it's pretty easy for me to locate that by going into "birds" then "grosbeaks" then "pine."
I'm sure it's not how everyone else does it. Just seems to work for me. I shoot RAW +small jpeg, and sort on my computer by double clicking the small jpeg, which opens instantly, and deciding to keep or cast away the pair of images based on what I see there. I don't use any RAW download software, just pulling the images off my cards and loading them into each folder.
I've read the DAM Book, understand Krogh's reasons for using dates in file names, but decided to stick with the codes I was using with slides for one important reason: editors say they like short, meaningful names as opposed to long, meaningless ones. So far (two months), the system seems easy enough. RAW files are kept in folders by date taken, but the file names themselves do not include the date; instead they are in the format Beittel_X-ABCD-123, where X denotes mammal, bird, reptile, U.S. scenic, World scenic, etc., and ABCD is a code for the species or place (this code grows out of the banding codes widely used for birds). Bridge allows batch renaming of images in this format, though it's up to me to make sure I'm not creating duplicate file names by consulting my IViewMediaPro catalog of existing RAW files. Most requests for my images are by species, and with this file structure, species appear alphabetically in the IViewMediaPro catalog, just the way my slides are filed. I'm keyworking all my top-ranked images, so they can be found through searches on location, behavior, season, etc.
Steve brings up a very good point, and that is doing what works for you, not necessarily what someone else says is best. When I started labeling and filing my slides, I spent a lot of time coming up with a system and asking myself questions like "How will I be able to find this in 5 years?" or "What if someone sees this image in a magazine and calls me several years after it was published and wants to license it. How will I find it?"
Right now I'm on my third incarnation of a digital filing/naming system and I think I have it the way I like it. The beauty of digital is that it's a lot easier to change systems than with slides.
Cliff mentioned editors wanting file names with meaning and that's a great point. I embed my original file name in the IPTC Title field in metadata. That way I can go ahead and submit images with different names while at the same time preserving a reference to the images "real" file name.
I generate my file names the way I do because I don't want to have to refer to codes or next available numbers. I tried that already and for me, it was just too burdensome. But for others it works.
One great thing about digital is that you could conceivably create copies of images (maybe just a few hundred to start with) and try out different naming/filing systems. Just use a test group (keep the originals in separate folders and don't mess with them). Then go ahead and play around and it won't matter if you mangle file names or otherwise abuse your images. Try out the different programs like iView or Portfolio or Digital Pro, they all offer trials.
when you've come up with a system that works for you, then implement it on your protected originals.
No, you don't have to have a cataloging program, they just make life a lot easier.
Bridge is not a cataloging program, it's a browser (though in CS3 the new filter tab in Bridge is very nice for sorting and separating within a folder.
The cataloging programs allow you to create virtual folders of images without the need to copy the image file over to a new folder. In a cataloging program, you have access to information and sorting options for all of your images at once. In Bridge, you're basically stuck with one folder at a time.
A cataloging program allows you to create custom metadata fields. For instance, you could have a field for print prices or a field that lets you know which agency you've sent the image to.
You have more versatile output options with the catalog programs too, from nicely designed web pages, to better looking contact sheets, to burning cds or dvds with the images and a catalog reader, which the editor (or whomever you've sent your images to) can use to perform searches and sortings.
Search functions in a cataloging program are also more robust.
When I was working on a book last year, I used custom fields in Portfolio to tag images for certain sections of the book. Some images got tagged with more than one tag. But I was able to look at my sets of images very easily. When it got down to the final choices, I could bring up each set, click on the images I planned on using and then instruct the program to place a copy of the images in a new folder, which would then be used for burning cds.
Think of it this way. Say you have a bunch of slides and you want to have pages of slides for a certain project. But some of those same images will also be used in another project. To have all images in all projects you'd need several slides of the same image. Or you'd have to make dupes.
If doing something similar with Bridge, you'd need to make copies of images and drop them into project folders. With a cataloging program, you simply apply a keyword or a term in a custom field and sort by those. In iView, you can create also Catalog Sets that are really virtual folders.
When you place an image in one of these folders, you're really placing a referrence to that image and every time you look in that set, you're just looking at a thumbnail and all the metadata that's in the database of the cataloging program.
If you tried to do all this in Bridge, you'd need a copy of the image in each project folder and that just takes up extra hardrive space and extra computer resources.
In short, you have many more searching and sorting options that can be done a lot faster in a cataloging program.
Peter Krogh, in his The DAM Book makes this point a lot better (and probably with more brevity) than I do.
Thanks for the explanation Rod. I have downloaded trial version of downloader pro and will try it tonight at home. Actually am looking for a good 'archive process' and understand the importance of it. Just I never really did it. I shoot very few images and have been cataloging 'manually' till now. I can imagine that it is quite a task if your are shooting lots of images. I see it coming.
Thanks.
I think what works best has a lot to do with how many images you have and/or how frequently you shoot. I agree with Steve and Rod that the best approach is to take the best of what you've read and heard, and use it to create a system that works for your shooting habits, volume of images and way of thinking.
I use a folder system rather than a file-naming system. I don't care what the file name is, only that the system is logical to my way of thinking and I can find a specific image quickly. My folders are set up as shown below (wish I could upload a screenshot). I have a separate but identical system for film scans.
This works well for my shooting approach, which has me keeping maybe a 100 images from any of my 5 or 6 significant trips each year. The folders allow me zero in quickly on a desired shoot, and findng the desired image - by number or visually via thumbnail - is easy for me. Within any given folder, I simply use the filename as defined by the camera (e.g., DSC_1189) which spares me the pain of renaming everything. When I put an image on my website, I usually use a truncated version of the filepath name (e.g., 2007_CO_DSC_1189) to point me to the right location when I get a print order.
If I were shooting fewer images over a much higher number of locations, this approach probably wouldn't work well. Or, if I shot at Holy Cross Wilderness frequently bringing back thousands of images each time, and wanted to be able to collectively view all images from every shoot at that specific location, this approach would get messy.
I started with a geographically oriented folder/file naming system essentially identical to Rick's, and I also keep the origiinal file names generated by the camera.
Under each location, the image folders are organized by date (Yr-Mo) of shoot:
\Colorado\SanJuanMtns\SneffelsWilderness\BlueLakes\2007-04\"camera filename"......
I feel this organization will let me find an image quickly with a simple browser, such as Bridge, even if my database application totally craters, and it will allow me to move to a new database application very efficiently, if needed. I'm very concerned about database corruption, but I'm REALLY paranoid about being at the mercy of ever changing software applications.
Recently I've catalogued everything with Lightroom keywords and tags. It is wonderful being able to search across folder structures instantly! I set the default to store all of the metadata in sidecar (XMP) files, so the image database could be quickly rebuilt from scratch, or transterred to a new application, if needed.
A geographic organization works well for landscape photographers. It may not be best for someone specializing in birds. Also, the DAM book makes a good case for a date based system and "buckets" which allow for an efficient archival process. However, I much prefer to keep my RAW and derivative files together in a geiographical file structure.
Even though there may not be an obvious system that works for everyone, it sure is worthwhile considering what others are doing and what is best for your needs and habits. Also, assume that those needs WILL change over time.
We include dates and original file #'s along with subject or location. That's because we have been shooting for some of our clients for years and a chronology can be important. The habit has spilled over into our nature shooting, and on several occasions the dates have proven important for documenting changes or events in specific locations. For example, we have (had) a large sea arch on the island in front of our house, and I've photographed it for almost 30 years. It collapsed in a minor earthquake three years ago, and by comparing my photos chronologically the geologists were able to trace the small changes that led up to its fall. Too much variability in lighting, framing and season to assemble them for a "time-lapse" effect, but still interesting to compare.
I note the same kinds of changes in vegetation distribution in some areas of Alaska over 30 years, which some scientist point to as signs of global warming. Those dated shots might be really useful someday to my grandkids-turned-scientists. Heck, looking at my photos of popular landmarks in Southwestern parks over the last 15 years, it's easy to see the effects of more foot traffic.
Aren't we all shooting for "posterity," as well as our own fun and profit?
Hank, since the dates are stored automatically as metadata, what benefit do you see in having a date as part of each file name? I do have dates as part of the folder name, so that shoots are sorted sequentially.
Of course you probably got into the habit back in the days of film.
I simply don't want to dive into the metadata to get it, especially with lots of older files. It also reflects our archiving system with older files and culls onto discs. Discs are labeled with date, subject and image # range. The allstars and keepers make it to the digital filing system, but all culls (along with original capture of the keepers) go on disc and into file cabinets. You'd be surprised how often clients pay for a selection of images on one occasion, but years later remember others from the shoot and want to revisit the original shoot and buy additional or alternate images. That's especially true with changes in personel, insurance claims, and historic applications. Like I said, it's a habit with us. But we are rewarded often enough to keep it rather than change. As in the examples I cited in my last post, it's also starting to pay off in the historic aspects of our natura shooting. Does that make me old now?
What cataloging program do you guys suggest for working with large tiff files?
I gave the trial of iView media pro a go but it was horribly slow with files of 100+mb and it wouldn't show tiff's compressed with the zip encoding.
Anny toughts? Thanks.
Discussion
17 repliesThanks lots for input!
I started doing this quite a ways back when there weren't a lot of commercially available databases (the couple that were available were ridiculously priced) so I ended up writing my own.
Guy
I'm pretty much following Peter's suggestions as outlined in The DAM Book. For my file naming, I'm using the following format: Barbee_x_WWYY_1234.dng (or whatever the extension happens to be)
where x is a camera code (just to keep from the possibility of duplication of file names and to quickly id which camera was used-in my case either the D100 or the D200), WWYY is a date code for week and year (instead of the exact date of capture--this saves two digits over using the full date), and 1234 represents the number from the original camera file name.
By the way, using date of capture in a file name is another way of minimizing the possibility of duplicating file names. Remember, your camera will count up to 9999 and then start over, so you could easily start duplicating file names. And if you're shooting with two cameras, it's even easier to duplicate names.
For me, this is easy because the file names are automatically generated when I download the images using Downloader Pro.
I used to keep all my files in a folder hierarchy, maybe something like this: Nature and Travel/Locations/National Parks/State/Olympic/rainforest. But this got way to cumbersome when it came to filing images. Many times, an image could easily be filed in two or more folders and I was wasting time in figuring out which one is should go into.
Now, my originals (DNG raw files and original jpg captures) go into "original" folders such as orig_001, orig_002, etc. Each folder is about the size of a dvd and all folders are backed up to two other hard drives (one goes off site) and two sets of dvds (one set off site)
Any derivitives (images processed for prints, etc.) go into folders marked drv_001, drv_002, etc.
the images in these folders are, because of the date code and file number, automatically arranged in ascending order, earliest to latest. This can sometimes be an advantage.
The trick to this is to apply metadata to the images as soon as you possible can. In Bridge, I add caption and location information and any keywords and ratings and labels (I'm not as good at applying keywords as I should be). Using templates in Bridge can make this go very fast.
After I've deleted the not-so-good images, I convert all the keepers to the dng format and then use a utility called Rank and File Lite to embed labels, ratings, and the filename into the metadata. I then move these keepers to their final folder (in this case the next available orig_xxx folder)
The images are then cataloged in iView Media Pro where I can search by just about any parameter I can think of. There's even a date finder that makes finding things by exact date shot.
In iView, I've also set up Catalog Sets that I can drop images into for sorting in other ways, like if I want to look at all my rainforest pictures at once or if I'm looking images to use for specific teaching programs.
By using a DAM application like iView or Portfolio, you can organize your images in any way you want and you're not stuck with any one way. I can view images by date shot, by location, by state, by ISO, by EXIV data, by Catalog Sets, or just about any criteria.
let me know if that makes sense
As i found that folders with specific information to be difficult to keep up with, much like Rod above mentions. I use Image Ingester to remane my files which makesthe rename mindless (that is a good thing - my mind wanders and gets creative on mundance tasks which makes files difficult if not impossible to find!)
I just moved 5 years of digital files over to this system about a month ago (having tried Peter's system for about 6 months). The move process sent very well and I think I can find the raw images faster and easier than ever before. Now I just need to learn to enter metadata more quickly!
Scott
This is especially helpful when clients enquire which species they might expect during photo excursions, and because some want to target specific species. I have one couple coming early next month who are specifically interested in lupines, and by looking back in the last two years' folders, I can see that lupine blooms peaked on the first week of June both years, which helps me schedule the best day for the excursion.
Images that I process and tone stay in the dated folders, but copies get moved over to other folders labeled by species. I have a series of those. If I want to find a female wood duck, I go to the folder titled "birds" and into the "ducks" folder and from there to the "wood duck" folder, where my keeper wood duck images are. As I amass more keepers, I'll create more sub-folders, eventually creating "female" and "male" and "family" subfolders within the "wood duck" folder.
Most queries that come to me from the publications I sell to come as a species inquiry, such as: "I need an image of a male pine grosbeak on spruce branches," so it's pretty easy for me to locate that by going into "birds" then "grosbeaks" then "pine."
I'm sure it's not how everyone else does it. Just seems to work for me. I shoot RAW +small jpeg, and sort on my computer by double clicking the small jpeg, which opens instantly, and deciding to keep or cast away the pair of images based on what I see there. I don't use any RAW download software, just pulling the images off my cards and loading them into each folder.
Right now I'm on my third incarnation of a digital filing/naming system and I think I have it the way I like it. The beauty of digital is that it's a lot easier to change systems than with slides.
Cliff mentioned editors wanting file names with meaning and that's a great point. I embed my original file name in the IPTC Title field in metadata. That way I can go ahead and submit images with different names while at the same time preserving a reference to the images "real" file name.
I generate my file names the way I do because I don't want to have to refer to codes or next available numbers. I tried that already and for me, it was just too burdensome. But for others it works.
One great thing about digital is that you could conceivably create copies of images (maybe just a few hundred to start with) and try out different naming/filing systems. Just use a test group (keep the originals in separate folders and don't mess with them). Then go ahead and play around and it won't matter if you mangle file names or otherwise abuse your images. Try out the different programs like iView or Portfolio or Digital Pro, they all offer trials.
when you've come up with a system that works for you, then implement it on your protected originals.
Bridge is not a cataloging program, it's a browser (though in CS3 the new filter tab in Bridge is very nice for sorting and separating within a folder.
The cataloging programs allow you to create virtual folders of images without the need to copy the image file over to a new folder. In a cataloging program, you have access to information and sorting options for all of your images at once. In Bridge, you're basically stuck with one folder at a time.
A cataloging program allows you to create custom metadata fields. For instance, you could have a field for print prices or a field that lets you know which agency you've sent the image to.
You have more versatile output options with the catalog programs too, from nicely designed web pages, to better looking contact sheets, to burning cds or dvds with the images and a catalog reader, which the editor (or whomever you've sent your images to) can use to perform searches and sortings.
Search functions in a cataloging program are also more robust.
When I was working on a book last year, I used custom fields in Portfolio to tag images for certain sections of the book. Some images got tagged with more than one tag. But I was able to look at my sets of images very easily. When it got down to the final choices, I could bring up each set, click on the images I planned on using and then instruct the program to place a copy of the images in a new folder, which would then be used for burning cds.
Think of it this way. Say you have a bunch of slides and you want to have pages of slides for a certain project. But some of those same images will also be used in another project. To have all images in all projects you'd need several slides of the same image. Or you'd have to make dupes.
If doing something similar with Bridge, you'd need to make copies of images and drop them into project folders. With a cataloging program, you simply apply a keyword or a term in a custom field and sort by those. In iView, you can create also Catalog Sets that are really virtual folders.
When you place an image in one of these folders, you're really placing a referrence to that image and every time you look in that set, you're just looking at a thumbnail and all the metadata that's in the database of the cataloging program.
If you tried to do all this in Bridge, you'd need a copy of the image in each project folder and that just takes up extra hardrive space and extra computer resources.
In short, you have many more searching and sorting options that can be done a lot faster in a cataloging program.
Peter Krogh, in his The DAM Book makes this point a lot better (and probably with more brevity) than I do.
Thanks.
I use a folder system rather than a file-naming system. I don't care what the file name is, only that the system is logical to my way of thinking and I can find a specific image quickly. My folders are set up as shown below (wish I could upload a screenshot). I have a separate but identical system for film scans.
2007/Digital/D2X/Colorado/2007 D2X Holy Cross Wilderness/DSC_1189.NEF
2007/Digita/C-8080/Colorado/2007 C-8080 Holy Cross Wilderness/P0307125.ORF
This works well for my shooting approach, which has me keeping maybe a 100 images from any of my 5 or 6 significant trips each year. The folders allow me zero in quickly on a desired shoot, and findng the desired image - by number or visually via thumbnail - is easy for me. Within any given folder, I simply use the filename as defined by the camera (e.g., DSC_1189) which spares me the pain of renaming everything. When I put an image on my website, I usually use a truncated version of the filepath name (e.g., 2007_CO_DSC_1189) to point me to the right location when I get a print order.
If I were shooting fewer images over a much higher number of locations, this approach probably wouldn't work well. Or, if I shot at Holy Cross Wilderness frequently bringing back thousands of images each time, and wanted to be able to collectively view all images from every shoot at that specific location, this approach would get messy.
Under each location, the image folders are organized by date (Yr-Mo) of shoot:
\Colorado\SanJuanMtns\SneffelsWilderness\BlueLakes\2007-04\"camera filename"......
I feel this organization will let me find an image quickly with a simple browser, such as Bridge, even if my database application totally craters, and it will allow me to move to a new database application very efficiently, if needed. I'm very concerned about database corruption, but I'm REALLY paranoid about being at the mercy of ever changing software applications.
Recently I've catalogued everything with Lightroom keywords and tags. It is wonderful being able to search across folder structures instantly! I set the default to store all of the metadata in sidecar (XMP) files, so the image database could be quickly rebuilt from scratch, or transterred to a new application, if needed.
A geographic organization works well for landscape photographers. It may not be best for someone specializing in birds. Also, the DAM book makes a good case for a date based system and "buckets" which allow for an efficient archival process. However, I much prefer to keep my RAW and derivative files together in a geiographical file structure.
Even though there may not be an obvious system that works for everyone, it sure is worthwhile considering what others are doing and what is best for your needs and habits. Also, assume that those needs WILL change over time.
I note the same kinds of changes in vegetation distribution in some areas of Alaska over 30 years, which some scientist point to as signs of global warming. Those dated shots might be really useful someday to my grandkids-turned-scientists. Heck, looking at my photos of popular landmarks in Southwestern parks over the last 15 years, it's easy to see the effects of more foot traffic.
Aren't we all shooting for "posterity," as well as our own fun and profit?
Of course you probably got into the habit back in the days of film.
I gave the trial of iView media pro a go but it was horribly slow with files of 100+mb and it wouldn't show tiff's compressed with the zip encoding.
Anny toughts? Thanks.
Wim.