6 Responses to Features and Functions that my Digital Asset Management System (DAM) should have

  1. Per Karlsson August 11, 2014 at 12:52 #

    I had a comment on a forum that I think is so interesting that I would like to share it here:

    I have one quibble. For item #7 (keywords/captions) you say:

    “A note on hierarchical keywords: I do not need the DAM to handle hierarchical keywords. Hierarchical keywords, or a “controlled vocabulary”, can be a good tool in some situations to find good keywords. But hierarchical keywords / a controlled vocabulary is too rigid a structure for it to be applicable when keywording images in a DAM.”

    I’m not sure what you mean by this statement. A controlled vocabulary is not the same as hierarchical keywords. As I understand it, a controlled vocabulary is a managed list (or lists) of regular keywords, whereas (a set of) hierarchical keywords is a specific type of controlled vocabulary that lets you define relationships between keywords.

    These tools are implemented separately in Expression Media/Media Pro and both are essential to my use of the software (I actually still use Expression Media).

    I have found hierarchical keywords to be especially useful because of their memorized structure. If you’re cataloguing wildlife (named according to a strict scientific/taxonomic structure that can consist of a dozen keywords), hierarchical keywords can save hours of time when applied to future images.

    These are precisely the features I expect from a DAM.

    Here’s my answer:

    Yes, I agree, hierarchical keywords (HK) and a controlled vocabulary (CV) is not the same thing.

    And yes, I can understand that many people would find either or both a HK and CV very useful. I can also understand that for some it would be a critical component of a DAM.

    However, not for me, not for the way I work.

    I tried for some time to use David Rieck’s Controlled Vocabulary (http://www.controlledvocabulary.com/) and have also occasionally tried using HK. But I always found it to be too rigid.

    On the other hand I have used it as “inspiration”, but in a more flexible way, to help feed my own keyword lists.

    I think that i CV would be most relevant in a “closed” environment, within an organisation. But when you are working with “publicly” available stock the “controlled” part is too limiting. Instead you have to be imaginative. You have instead to try and think of “what possible search terms could people use and be happy to find this picture?”

    And HK is obviously useful when you work with e.g. botanical subjects with a clear hierarchy. But I generally don’t

    But then again, if I found a good DAM that had a CV or HK perhaps I would find a good way to use them. (BTW that is actually a feature that my current DAM, Portfolio, does NOT have.)

    • Chris Rakoczy April 2, 2015 at 17:02 #

      It’s my understanding that Controlled Vocabularies refer to *content* while Hierachies refer to *structure*.

      I am currently cleaning up a 3600+ word flat, messy, disorganized keyword list for my new employer. We have healthcare-specific keywords that would not likely appear in a generic CV dataset like that by David Rieck. In effect, I’m building my own CV based on the vocab we actually need. That only addresses the content, however.

      The structure is also important. I leverage Hierarchical Keywords to organize the vocab I am building into logical genus-specis type order, and also to apply multiple relevant parent keywords. For example, knowing where a photo was made is useful, so I have rooms and departments inside buildings inside facilities inside cities. I can apply Cityville > ABC Campus > North Building > XYZ Department simply by applying the most child keyword.

      In that way, I find that CV and MK are not the same, CV and MK are not mutually exclusive, and in fact CV and MK work hand-in-hand to make keywording more efficient and more robust.

  2. Per Karlsson August 11, 2014 at 13:59 #

    Here’s another very interesting comment that I take the liberty to copy from a thread on the excellent Media Pro user-to-user forum (http://goo.gl/bfL0ss)

    Interesting indeed. I follow all the dam tools and it will be very interesting how you compare them.

    I also have a few notes. Right now I can’t reply in detail but some short ones: you have the company as your first criterium. I think this is meaningless. We’ve seen enough of the big companies dropping their tools without notice. This includes Adobe, Microsoft, Apple (recently), extensis (recently), canto (recently), Corel (recently), and more. Maybe a valid aspect in this area is how long a product is in the market. Anything else is “air”.

    Also, your criteria are based on existing features here. For example; how you describe the portfolio “categories” is not what I would call virtual galleries, though you could use it for that purpose.

    Your ideas about embedding metadata are outdated and more based on the limitations of Portfolio.

    Export to CSV etc; your argument is that export and import is a requirement to be “the only way to guarantee independence” is also outdated. How you describe it for Portfolio this does make sense for you to get your data in your new DAM, but once working with other dams you shouldn’t need its import. There are more, and more efficient, ways to achieve the same.

    Your search requirement is based on portfolio. While I agree with the requirement, I disagree with that outdated dialog to achieve this.

    Multiple catalogues and searching over them are a result of the limitations of Portfolio. Is that still 2Gb? Any modern dam can store up to terabytes in a single database. Searching over multiple databases, at least to me, seems like you shouldn’t have separated them in the first place. But I do understand that if a dam only supports 2GB databases that this is a requirement.

    It’s not my intention to criticize your approach, but I want to illustrate that some requirements are not always relevant or my be open for different interpretations. Good luck with your comparisons. And don’t forget; the devil is always in the details with these tools and getting to the details takes lots of time. And they change with every update of the software.

    You are absolutely right, my comments are based on my habit of working with Portfolio. Inevitably. In time, after changing, the way I work may very well change.

    I don’t agree with your comment on “company”. It is a matter of taking an informed decision. But I still think it is important to know a bit about who’s behind it. Is it a big company, famous for its poor customer support and bullying? Is it a small one-man shop? It is impossible to predict the future but it is better to know something about the current situation.

    Also, I am not saying that I want a new DAM to do exactly what Portfolio does, I am just trying to find ways of performing the tasks that I need to do in a DAM. Inevitably I use terminology from Portfolio since that is my current tool. For example the “virtual galleries” is a tool that significantly speeds up my keywording. (If Portfolio’s “categories” were intended in another way is beside the point. What is important is what it allows me to do.)

    I don’t see why “Export to CSV etc; your argument is that export and import is a requirement to be “the only way to guarantee independence” is also outdated.”. Unless you rely 100% on embedded metadata, or on sidecar files I don’t see how it can be outdated.

    Also, it is an important tool for me to provide data to Alamy. I don’t see how I could do that in any other way. I’d be happy to learn!

    Your other points on limitations in Portfolio and that it is outdated (it certainly is!) may or may not also be relevant for other DAM. Remains to be seen.

    I definitely agree with your conclusion, the devil is in the details. It is not sufficient to look at specs or have a quick run-through of the SW. You have to go through all the workflow process in detail and see how it works for you!

    I’d be curious to know what you suggest today as DAM!

  3. Ashia October 15, 2014 at 23:45 #

    Question #1: I would be interested to hear whether you or anyone else has considered Filemaker Pro 13. It is a generic database software that can be customized for a variety of purposes. I am trying to transfer files from Portfolio 8.5 to another database system and it is proving to be quite painful. So far I have been able to export the Portfolio data and import it into Filemaker, but the images don’t transfer, so that is really not very helpful! I am quite distressed that Extensis pulled the plug. It seems like *some* company out there should want to scoop up those customers who just want an easy-to-use stand-alone (not cloud-based!) DAM…
    Question #2: With the various DAM solutions you are exploring, I am wondering about the ease of transfer from Portfolio. Any thoughts/experience regarding this?
    Thanks in advance for your helpful post!!!

  4. Sherwood Botsford December 30, 2016 at 06:53 #

    I too am in the position of trying to find another horse in the middle of the stream.

    Yesterday I published this on Medium.
    Photo Management

    Apple announced that Aperture was not going to be maintained. Old news. Of course all work on it stopped, and all plugin development stopped.

    Now, Aperture wasn’t the bee’s knees by any stretch of the imagination. But since then I’ve been searching for a replacement. Much like a Swiss army knife, it did many things in a convenient bundle, but didn’t do any of them really well.

    So far all I’ve found, to carry on the metaphor, are sheath knifes, dremel tools, scalpels, and empty cardboard boxes. Lightroom comes closest — it’s more of a leatherman. Does lots of things, but uncomfortable in the hand.

    Here’s the dilemma:

    Most of the alternative products proposed are for editing images. Photoshop, Lightroom are the big boys, Affinity, Photo Supreme, Aftershot Pro are candidates.

    But editing is no good if you can’t find the image to start with. My situation: I have a tree farm. I try to take at least 100 pictures a month, to be used for catalogs, mailouts, blogs, newsletters. On occasion I have picked up my camera and gone out to the field, because I couldn’t find the pic I wanted in my archives. I don’t edit much. But I need to be able to find what I took 10 or 20 years ago.

    Photo Mechanic is a good tool in the chain: It’s probably best of breed for keywording, rating, captioning, and culling. It’s a decent browser, but it’s no good as a search tool.
    Lightroom has moderate ability in keywording — interface is clunky, but it understands hierarchies. Haven’t really tried it yet for searching. LR has a 7 day trial, which is hardly enough to find your way.

    I’ve found that there are four distinct tasks to deal with once you have more than a few thousand photos.

    1. The ability to add metadata: Keywords, captions, descriptions. Computers don’t understand, “Find me that pic I took of maple trees on a sunny winter day, around Christmas time.”

    2. The ability to search for images by using that metadata. Now I can find keyword: Maple trees; Date 20 to 30 December. Search is implied with Smart Albums — Albums that you don’t add images one by one, but ones that are based on metadata. E.g. “Last 4 weeks” “Spring Inventory” “Facebook Shares” The first is based on image dates, the next two on keywords.

    3. The ability to edit images, both destructively, or non-destructively. A non-destructive edit eventually has to be turned into an new image to be used outside the PMD. This does not have to be handled by the PMD, but until I get a camera that does what I mean, instead of what I told it to, I will need to edit images.

    4. The ability to track multiple versions of a file. Those non-destructive edits have to be converted to a new image at some point if they are going to be used. Nobody tracks the offspring at this point.

    Photo Mechanic excels at task 1. It understands hierarchies — that “maple” may be a tree or a flavour of syrup, or cabinet wood or even (shudder) a formica pattern. It understands synonyms: When you label something Maple (tree) it can fill in Acer, the botanical genus name. It will also put in the parents of the hierarchy.

    Aperture isn’t bad at task 2, although you start having problems if you want to do things like “Show me images that would fit in a 1024 pixel box.” as when you reduce that to exif pixel searches, you need both OR and AND in your search, which Aperture doesn’t do.
    Editing: See the list above. There are others, but they are either very cheap and very limited, or they cost your firstborn son, or they are Software as a Service, and require a monthly pint of blood.

    Versioning: Not there yet. Not really. Both Aperture and Lightroom implement “Stacks” where you can put in related images — You have the continuous shooting going on, so you have 15 pictures taken in 3 seconds. Lump them together. Nice first step, but a long way between a toddler’s first step, and running a marathon.

    — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

    In different circumstances you will want to use different tools, the Aperture habit of slurping everything into a managed library isn’t acceptable. The pictures have to be in the file system where you can find them, not hidden away.

    But this exposes your database to mangling by someone moving or renaming files. Your Photo Manager Database (PMD from now on) thinks that the maple shot is in /Path/To/Pix/Masters/2016/12/25/DSC_1293.jpg. If you move it or rename it, then the database can’t find it. This is not an unsolvable problem. I’ll talk about two ways:

    A: FSwatch is a program available on unix, linux, mac and windows for monitoring file system changes. It can tell you that a file was moved, renamed, opened with a particular app. There are others out there I’m sure.

    Still, this might not be enough. The moves may come too fast. The watcher program may crash. The PMD may not be paying attention to the Watcher.

    B: The second way is more complicated, but it also solves problem 4 above.

    Almost all image files support some types of metadata. Most are writeable, although some with limitations. (E.g. some raw camera formats allow you to change data, but not to change the length. Change Dog to Cat, but not to horse. Horse makes it 2 characters longer. Fortunately there are usually blank fields that can be used. This will be one of those “devil in the details” problems that software designers are paid for.

    Exiftool, ImageMagick, and Exiv2 are tools that can read and translate this stuff, so developers don’t have to reinvent this particular wheel.

    Another useful idea is that of a checksum. This is at heart a complicated math function that uses all or part of a file as input, and produces a stream of alphabet soup as output. It’s repeatable: Same file chunk in, same soup out. Change 1 bit of the input, get new flavour of soup. Chances of 2 files having the same checksum are billions to one. A checksum is a fingerprint, unique to the file, typically 32 bytes to 128 bytes long

    Let’s see what we can do with these tools:

    Compute a checksum, such as MD5 on the image part of the data. This gets done by whatever program you use for importing the image. These are 2 images — shot 3/10 of a second apart, in jpeg and raw (nef). Entirely different checksums.
    MD5 ./2016–12–25_14–52–23.20.JPG = 2d501ae28886ba3e17ef00f96cb321ac
    MD5 ./2016–12–25_14–52–23.20.NEF = 75205c6c565a367b309953aea39fa1db
    MD5 ./2016–12–25_14–52–23.50.JPG = b1c41a9a9f93cf612415ff7beffb0496
    MD5 ./2016–12–25_14–52–23.50.NEF = d50d822d1efdecd4f3c1c9da32f522f1

    Now we are going to make a habit of keeping a few metadata fields in all of our images. These will checksum just the image part of the file, not the metadata part. Otherwise the checksum would change every time you added a keyword.

    Original Image Checksum (OIC) when we bring an image into our PMD — Unless it already has this field. This field may be written into one of the IPTC fields that are largely unused, by setting a keyword or by hijacking some other field. Keywords clutter up user space, and so should be avoided. The @ will put it at the end of the list, so it’s likely to be ignored.

    Generation checksum (GC) This isn’t really a checksum — take the OIC and add a 2 character string to it, starting with aa. Do another one from the same master image, and it gets ab added, third one ac. If you prefer, use a 3 digit code.

    We can add timestamps to all of the above if we wish. But keep it simple for for now.

    Let’s watch this in action:

    We start with 2016–12–25_14–52–23.20.JPG and record the key for it @OIC=d50d822d1efdecd4f3c1c9da32f522f1

    We want to post a copy of this onto facebook, so we make a copy that will fit in a 1024×1024 box. This image is given a GC=d50d822d1efdecd4f3c1c9da32f522f1-aa Later we decide to make a black and white version of our original image. This will get GC=d50d822d1efdecd4f3c1c9da32f522f1-ab Suppose now we shrank this one so that I could use it as a thumbnail on a forum. Now it’s been made from a 2nd generation imge so it gets GC=d50d822d1efdecd4f3c1c9da32f522f1-abaa

    Now you have a way to identify a file even if it gets renamed or moved. Your edits are keyed to this internal name. Developers need to write a “Get Image Identifier” or “Who_Are_You” function for each type of image. A “Reconnect database to images” basically scans the image folders and pulls the ID string, and compares that to it’s internal records. This will update file moves and renames that somehow got past the Watcher (Eg. You weren’t running PMD when you moved the file.)

    When we created these new images we can put them either inside the managed folders, or outside. If they are outside, PMD ignores them. You may initially want them outside for your facebook upload, then later move them back into a ‘facebook’ folder that’s managed.
    Watcher sees the new files, and notifies PMD of their existence. PMD reads the OIC, looks for a GC, and if found,increments it according to what it thinks is the next unused letter pair. If not found creates one. If PMD was the program that created the file, the GC is unchanged. (Hmm. May need another metadata field to track status of this. Back to the drawing board.)

    PMD at some point asks you about metadata for the new file. Depending on the programmers you may have options to “copy metadata from original, or some combination of certain fields. E.g. if your mangling was to photoshop the 3rd bridesmaid out of the image, it should no longer have her name in the people keywords. You may want to add the keyword “Facebook Album”

    You may also have rules for folders: If you put a file in this folder, it gets tagged FaceBook.
    But anything that creates new copy of the image on disk gets a new Generational Checksum. Two images that have the same GC should be identical.

    This will overall allow multiple programs to be used on a folder-tree of images without shooting yourself in the foot. Now you can use any combination of image editors, image browsers, metadata writers you wish.

    It also means that if you add a keyword to an image you can set your PMD to write it to ALL images that derive from it.

    Want to write sidecar files instead of mangling the original files repeatedly? Just get the ID in the original once, and now sidecar files can carry that info. If they get separated from their image, it’s easy now to reconnect them. Worst case: Calculate the checksum on any reasonably large chunk of image data in the file, stuff that in a database,then to rebuild your database, repeat the calculations. This means you don’t write at all into the file (Necessary if using read only media) But rebuilding your database now will take a lot longer.

    The only requirement is that any editor has to at least preserve the existing ID metadata. Writers of the PMD software would certify which editors they know behave nicely. The test is simple: Here’s a set of files. Edit them, and return them. Good software could go belt and suspenders, and use multiple strategies so that the metadata can always be reconnected to the right file.

    This form of software has another advantage: It makes proving that you are the guy who took it a lot easier. You shoot raw. You send them Tiffs. That tiff will have a different image checksum, but it will show the OIC. If they claim ownership, ask to see the original image. If the only copies of the original image are in your hands, they will have a tough time coming up with an image that looks like yours, and has the right checksum.

    The GCs won’t be unique across the world. If I give you a copy of my image, then you edit it, then your PMD may assign a GC that my PMD has already used. If you want to be world wide unique, then you need to glom on another full checksum with each generation.

    Now, while I’m waiting for someone to invent this, anyone know of something that comes close?

Trackbacks/Pingbacks

  1. I am looking for a good DAM: Digital Asset Management software | BKWine Photography | - August 10, 2014

    […] Here is a follow-up post where I look more in detail at DAM requirements. […]

Leave a Reply

Subscribe to comments:

Notify me of followup comments via e-mail. You can also subscribe without commenting.