DNG, RAW and JPEG: What I Use & Why

Hello there! If you are new here, you might want to subscribe to the RSS feed for updates on this topic.

Almost 5 years ago I wrote an article on the pros and cons of using RAW versus JPEG and why I was shooting in RAW (see RAW vs JPEG: Is Shooting RAW Format For Me?). These days the argument of using RAW versus JPEG is well… not an argument at all. Most photographers realize that shooting in RAW provides a great number of benefits and many if not all the reasons to avoid using RAW have been nullified thanks to a variety of technical innovations including larger capacity compact flash drives & hard drives, maturation of RAW converters, development of photo specific editing/management software and faster more powerful computers. Now the question that faces photographers is should I convert my RAW files to DNG?

What is DNG:
DNG (digital negative) is a public archival RAW format developed by Adobe Systems Incorporated. The intent of the DNG format is to provide a common archival standard for software developers and camera manufacturers. DNG has been integrated into a variety of software applications, beyond Adobe products such as Lightroom and Photoshop, and is currently supported as a native file by several camera manufacturers (see DNG Supporters ). Proprietary RAW formats from cameras that do not support DNG natively can be converted via Adobe’s free DNG converter application or through well known programs such as Lightroom, Photoshop, Aperture, GIMP, etc.

Pros and Cons of DNG:
Pros:

DNG files offer you the option to embed the original RAW file with in it
Note: The RAW file can be extracted via the DNG converter application
DNG files often are smaller* in file size than the original RAW file
(* less the embedding of the original JPG or RAW file)
DNG is a universal format addressing the issue of non-standard RAW formats between the various camera manufacturers.
Note: Adoption of the DNG standard enables files from new cameras to be immediately supported by photo editing software.
Adobe software enables auto conversion upon importing of image files
DNG files can have the metadata written to it versus to a separate XMP sidecar file simplifying file management
Note: This minimizes risk of XMP sidecar file loss or worse the impact of the corruption/loss of an Adobe Lightroom database)

Cons:

DNG is not a native format from Nikon or Canon, two of the largest camera manufacturers
DNG file conversion does take extra time
Release of new proprietary RAW formats could delay ones ability to convert RAW files to DNG until Adobe software updates are released
There is a risk, albeit very small, of camera manufacturers no longer participating in Adobe RAW/DNG programs
Corruption of DNG files during the conversion process
DNG files that include the original RAW file do take up more space than the original RAW file alone.

Why I’ve Converted to Using the DNG format:
I’ll be honest upon the announcement of the the DNG format in 2004 I never saw the value in using it. DNG seemed to be an unnecessary step that would add more time to my workflow and complicate my backup process. Not to mention back in 2004 there was uncertainty as to whether the DNG standard would be widely adopted.

Earlier this year DNG has officially become a part of my digital photography workflow. Upon import of my images to Adobe Lightroom I convert all my RAW files to DNG. Conversion of my RAW files to DNG happens before anything else. Only after the conversion do I begin my post-processing and keywording. Why the change of heart toward DNG?

My acceptance of the DNG file format was one of necessity. Two things happened that forced me to rethink my workflow:

I had a hard drive failure where I fortunately avoided catastrophic data loss (See A Happy Ending to My Drive Crash)
I picked up a new laptop to work on when in the field and needed a solution to easily synch metadata, keywords and image adjustments to my primary Lightroom database found on my office workstation.

Both of these issues were easily addressed by adopting the DNG file format. With metadata, keywords and image adjustment being written to the DNG file it alleviated the need for complex synchronization of RAW + XMP files and/or Lightroom database files. Focusing solely on synchronizing the DNG files knocked out two birds with one stone.

Lastly if I’m ever faced with a request to provide a RAW file as part of a photo licensing or publishing request I know with a DNG file my contact/licensing info will be embedded in the file versus in a loose XMP sidecar file.

Additional Questions & Thoughts:
Is DNG conversion of RAW files something I’d recommend?
Now that I’ve had a chance to work with DNG files in my revised workflow I think the answer is a solid yes. The DNG standard continues to gain momentum and the conversion process & options are robust enough for most everyone.

Does time to complete DNG conversion take long?
Conversion of RAW files to DNG does require extra time, but if you have a relatively new computer the time needed to complete the conversion should be minimal.

Is the conversion process perfect?
Unfortunately it is not. I have had situations where DNG files were corrupted upon creation. This has been a rare occurrence, but it does happen. It’s for this reason that I do keep a temporary copy of all my RAW files until I verify that the conversion has happened with out issue. Simply repeating the conversion of the file in question is enough to remedy the situation.

Does metadata get written to the DNG file automatically in Lightroom?
No, to write metadata to the DNG files from the Lightroom library you’re working in you must select all files you’d like to modify and select “Update DNG Preview & Metadata” under the “Metadata” menu.

Do I still backup my Lightroom library now that I use DNG files?
Yes I back up my Lightroom library both to a second drive on my computer and to a USB thumb drive I store off site. I do this to preserve metadata, keywords and image adjustments to older files found in my library that have yet to undergo DNG conversion. I also do this to save time in the event I need to restore my Lightroom database if the original is ever corrupted or lost.

Links:
Adobe Digital Negative (DNG) – Adobe.com

DNG Supporters – Adobe.com

DNG vs RAW Space Savings Chart – digitalstory.com

Bridge & DNG Workflow – DAM Book by Peter Krogh

Real World Camera Raw with Adobe Photoshop CS4: Using Adobe DNG Converter – Bruce Fraser and Jeff Schewe

Technorati Tags: Photography, Software, DNG, RAW, JPEG, Adobe, Metadata

Stay Connected with Jim
	Join Me On Twitter	Become a Fan on Facebook
	Listen to EXIF & Beyond	My Latest on YouTube

Posted by Jim on 08.10.2009 at 12:09 am
Tagged: Camera, Photography, Software, Technology , Adobe, Metadata, Photography, Technology

Related Posts

Juan

Nice article, just one comment. In the Additional Questions section you state that the metadata is not automatically updated in Lightroom. This is actually a setting that you can change under the "catalog settings" in the Lightroom dropdown menu, you can chose to have lightroom automatically write metadata to the DNG

Alexander

Really interesting blog post.

Was just wondering how important embedding the RAW file within the DMG file is. For the professional photographer is there a need to keep the RAW file for any reason other than paranoia/nostalgia when you can have a smaller DNG file as an alternative? For example, is it required by some clients/employers?

Also I'd love to know what settings any of you recommend for converting as the converter offers different JPEG preview size.

Thanks all.

jimgoldstein

Really the settings should be customized to your need. If you have a fast computer or a smaller library then having larger previews may be the way to go. If you're working with a slower computer and a huge library then you may want to use smaller previews.

I'm rather bummed that Lightroom doesn't let you embed the RAW file during the conversion through it. Only the DNG converter does this.

I keep the RAW files out of paranoia. I'd like to come up with a better reason, but I don't have one. In the event any corruption happens I prefer to have the original RAW file to work back from. Thanks for the comment.

Nathan Chapman

DNG files that include the original RAW file do take up more space than the original RAW file alone.

I'd just have to say that, for my experience with my Canon 1000D the 10.1 Megapixel images from it are actually smaller once I convert them from .cr2 (or something) to .dng

Having said that, I'm not embedding the original RAW file in it, I'm doing a straight conversion.

Also, I only have the one computer with a rather small hard drive, so this space saved does make quite a difference to me (up to about a third more photographs in the same amount of space)

Jonathan

Just a note: Converting to DNG is the workaround when one has a new camera but old software. I just upgraded to the Canon 5D Mark II, which produces .CR2 files, but still have Adobe CS2, whose Camera Raw has not and will not be updated to support .CR2. The solution: convert to DNG, then process the raw files.

Hornblade

Fwiw, the developer of 'Raw Photo Processor' recommends *not* using DNG. Check out the posts by Andrey 'hardloaf' in this thread:

http://forum.getdpi.com/forum/showthread.php?t=...

Riku

"Raws are actual Originals and results of Raw->DNG conversion are only inexact replicas."

Tough words from Mr. Hardloaf, and no sources to back him up. I'm sticking with DNG.

Martin Gommel

Hey Jim, good post here - kinda inspiration for maybe a follow up from my side. Let's see. Good to see you blogging!

Andre

But do you have the right algorithm to make a solid transition from a NEF file to a DNG file? Will it display like a jpg image?

Only nikon's software does the perfect read of these files. I think that when you make the change from NEF to DNG there is color errors. Camera Manufacturers make proprietary files so that se need to use and buy their software...

Chris Brown

Jim,

Another one here who hasn't converted, mainly through intertia, but partly because I'm a paranoid sort and would like to keep the original RAW data anyway.

If I converted to DNG with embedded RAW, I'd feel like I'd gone through a conversion process for the sake of it, and being a bit old-school in computing, a conversion is a chance for loss of fidelity, either through format design or by accident (software bugs etc). I'd keep the actual RAW files as they came off the camera anyway, so having that plus a DNG that also contained the same data would seem like a waste.

There is a long-term access argument - what happens in years to come when software doesn't process Canon's CR2 or CRW formats? That time may be a few years off, but when we're on Windows 18 or OS XVII or whatever, running Lightroom 8 or CS9, will our "legacy" format files be lost? I can see the potential benefit of DNG for that kind of situation, as hopefully it would be supported in some distant future.

Unless it's been replaced with DNG2, DNG3 etc...

I could convert to DNG and write those to recordable media (DVD or Blu-ray etc) so I have an archive format that's possibly more future-proof, but then what if computers of the future no longer have DVD or Blu-ray drives? Once the 3.5 floppy was ubiquitous, and before that the 5.25" floppy.

The whole subject of enabling future access to digital content of all types was something I was planning to blog out myself. I got as far as a draft with the first line as a reminder to myself, and no further!

Perhaps by the time I get to writing it, we'll be on Wordpress 4.x.x and the draft will be in an incompatible format?

Chris

latoga

Great write up Jim, thanks for sharing your experience and thoughts on DNG! I considered DNG a few years ago when I started working with LR1. At the time, the fact that the Metadata WASN'T in a sidecar file was one of the main reasons I stuck with RAW.

The reason being that I intended to use an online backup service to backup my master collection of photos. Since the NAS where those photos are stored is always on, I was concerned about the time needed to backup the archive of new photos after a shoot. e.g., I get back from a shoot, migrate all photos over to NAS; over the first night those photos get backed up to the online backup service; over next few days I'm working on photos and as I edit them and update the edits, the entire DNG file gets updated. This means that the entire (10-30GB) file needs to get re-uploaded to the online backup service. Versus just updating the meta data and having just a few KB being re-uploaded in the XMP sidecar file.

However, I never implemented the online backup part because most online backup services don't support backing up NAS devices (yet, or so they all like to claim). Though the logic of the backup issue still applies to any incremental backup of your photo repository...DNG will add time to that backup process.

I guess I never realized that all the editing data gets written to the DNG as well. This fact definitely helps speed up the "return from the field" computing time of exporting and importing LR catalogs from field laptop to digital darkroom computer. (Something I'm doing while I type this comment... :-) ).

I will have to test this aspect of DNG and see if it solves the problem of managing two LR catalogs and simplifying the sync process while making sure that any edits from the field aren't lost when I return to the home office. Even if it increases my backup times, that part is worth it. (Backups happen while I'm doing other work...like sleeping...anyway).

David Burren

Years ago I switched to using DNG files, primarily because iView (now Expression Media) shows the previews which ACR updates. As per Paul Dymond above. However, since I switched my main workflow to using Lightroom I've moved back to leaving the camera files in their native format. Less work for the computer to do, and because I'm keeping my Lightroom/Photoshop up to date it's as capable as the latest DNG Converter.

I see no disadvantage in having sidecar XMP files rather than embedded within the DNG files. In fact there's at least one advantage (as mentioned by JasonP). When the metadata is changed, only the XMP files need to get copied in the next backup operation (rather than re-copying entire DNG files). This can make a complete backup operation after hours of work simply a matter of a few minutes.
Also the XMPs don't "clutter up" the folders: Bridge, Lightroom, Expression Media, they all hide the XMP files from view and move/rename them as required just like other sidecar files.

Jim, you wrote "This minimizes risk of XMP sidecar file loss or worse the impact of the corruption/loss of an Adobe Lightroom database)". How?
The only "risk" is if someone deletes the XMP files, and if they're doing that they're probably as likely to delete other important files such as DNGs. The impact of an LRCAT loss is not affected by the use of XMP vs. DNG. DNG files need the same step of writing out the XMP data from Lightroom.

I always want to keep the original RAW files on-hand (whether I convert to DNG or not). This gives me the option of working with other RAW processors in the future. Not only the opcode example of the G1 mentioned above, but also things like checking what this image looks like in DPP. If I don't have the CR2 version I'll never know.
Incidentally, competitions such as the Wildlife Photographer of the Year competition (used to be the BBC comp) need the ability to check the original RAW for winning images (just as they used to need to see the original transparency). I think you'll find they only accept _camera_ RAW files, not DNG derivatives. So keep those originals around...

Jim

Thanks for the comment David. In dealing with 10's of thousands of images tracking XMP files is a royal pain. I'd rather worry about backing up one file type versus two and I'd rather not have to worry about 2 files vs 1 getting corrupted in the process of being backed up. To me more files equates to greater opportunity for loss or corruption. I can understand the speed of back up argument if you're focusing on XMP files alone, but for my situation its just not practical.

To your questions of "How?". There are a variety of ways files are lost or corrupted. In the 5 years I relied on RAW and XMP I have had several instances where XMP files were misplaced or errantly forgotten in the backup process. It happens. The more you have to juggle the more likely a little carelessness can have an impact.

My comment about the LR catalog was a reflection of restoring a LR catalog in the event it is corrupted or loss. If I had to start from scratch after losing a LR catalog my bet is that it would be faster to recreate with DNG files vs RAW+XMP. It could be the same, but thats my educated guess.

From the perspective of accessing files on multiple machines its far easier to track and move DNG files over... at least for me (note I'm doing so via eSATA drives for a faster transfer rate). My personal opinion is that DNG files with embedded metadata, keywords and image adjustments are a more streamlined approach. Having juggled, matched and sorted RAW+XMP enough times the process is old and tiring. Managing a single file per image is preferred.

No where in my write up have I stated I'm not backing up my RAW files. I'll continue to hold on to them until I become more comfortable with the RAW file being embedded in the DNG. I can see several reasons moving forward why I may opt to continue to backup the RAW file. The jury is still out on this. As you've mentioned if entering contests where you need to show the RAW files is important then keeping the RAW file will be a required step.

Thanks for the great comment.

David Burren

My own backups are done using filesystem-level synchronisation software that essentially mirrors whole folder trees. It takes care of the mechanics of identifying which files have changed and need copying, so for me there's no special tracking of XMP vs DNG files required. It just happens.

My comment about keeping the original RAWs wasn't directed at you in particular. I must admit there are a few years' worth of my images where I don't have the original RAWs, only DNGs...

Cheers!

Ivan Makarov

Jim -

I used to convert everything into DNG, but recently gave up as I noticed I can save a lot of time by not doing that when I am importing a lot of photos in one session ( you do notice the speed when you're importing 10+ Gb in one sitting). Frankly, I don't see a reason to convert anymore, and seeing you reporting that files may get corrupted during conversion process, I'll continue to stick with RAW format. Sometimes I come back to a RAW file weeks later, long after I erased it from my memory card, and while it is recoverable from the backup hard drive, it's a pain in the butt process I don't want to deal with.

While it's possible, I do not imagine Nikon or Adobe will ever discontinue supporting old RAW formats. However, if I was shooting with anything besides Nikon or Canon, I'd then for sure convert to DNG.

Jim

Ivan I think the intention is to have other manufacturers offer DNG as an alternate output rather than a replacement for their proprietary format.

Keep in mind out of tens of thousands of images I've had 2 or 3 images get corrupted on the first pass. 2 or 3 files out of 10's of thousands is hardly something to be concerned about particularly when its easy to test for.

My threshold of tolerance for DNG has increased because I have new very powerful computers that minimize the conversion time. This makes a huge difference and is one of the many factors that played into my adopting DNG. Something to think about. Perhaps after your next computer upgrade DNG might start to look a little more attractive. Maybe :)

Jay

Please forgive my ignorance, but how do I test the integrity of the DNG conversion from CR2? And what software options for integrity testing are available?

Can those tests be automated, ideally during the conversion process?

BTW, using LR 2.4 on Vista.

Thanks!

JasonP

Well, I'm one who WON'T be converting to DNG anytime soon. Specifically because I want the XMP sidecar file. Once my CR2's are backed up on a mirror drive, I leave auto-write XMP on in Lightroom and edit away. Keywords, metadata, geoencoding and developing get done when I have the time. With each edit, the XMP is updated. The next time I do a drive sync, the only thing I have to transfer is the XMP, not the entire file (as you would have to with DNG or RAW... the catalog is of course also backed up regularly). I can back up changes to a thousand files in literally seconds by only needing to transfer the sidecars.

The additional benefit is easily editing the metadata outside of Lightroom with exiftool or ExifToolGUI. When shooting with a few old manual lenses, I like to add in the EXIF to photos so that the info is available using LR's Metadata Browser. Writing to the XMP and reading the info back into LR means I don't have to write a new DNG when making changes, speeding things up considerably.

Until LR has full metadata editing capability and the ability to write individual bits to a file, I'm sticking with my sidecars.

kevwil

Exactly my perspective. I backup both locally and online, and I DO NOT WANT that large file being sent over the wire more than once. I've read the DNG spec - the embedded xmp metadata thing is an option, not a requirement. I would probably switch back to DNG if LR had an option to use xmp sidecar files with DNG.

Riku

Backing up DNG files isn't a problem if you use the right tools.

Blind copying of changed files WILL copy the multi-megabyte DNGs every time. But with better algorithms like rsync you can only move the changed parts, not the whole file.

Jim

Having avoided using DNG files for as many years as I have I can say definitively that there is no one right way to operate with DNG or RAW+XMP. If you find a system that works for you then who am I to say otherwise. I personally would not be able to operate that way and have the same level of success. More power to you and of course I hope the folks at Adobe track these comments to further refine the features they offer for both LR and their DNG converter.

Eric Johnson

Hi J.L. --

The latest Adobe DNG Converter 5.4 (and the corresponding Lightroom 2.4) supports the opscodes for the Panasonic G1 and GH1.

So you should be okay there -- if you use the latest Adobe tools.

Unfortunately, Aperture (and Mac OS X) does not yet support DNGs that have those opscodes.

So there are some work-arounds with DNG and Aperture:

http://elstudio.us/how-to-my-raw-workflow-for-t...

Jim

Thanks for passing on the info about the Opcodes Eric.

J.L. Williams

Oddly enough, I've just switched BACK to archiving native raw files after relying on DNG for several years.

The reason? Opcodes. These are the special tags manufacturers embed in their raw files that enable the manufacturer-supplied software to implement automatic correction of lens distortion, vignetting, color fringing, and other optical nuisances.

I never believed these were important until I got a Panasonic G1, and discovered that JPEG conversions from DNG files via Lightroom had a lot of troublesome artifacts that were absent from raw file conversions from the manufacturer's supplied software.

So now I'm stuck maintaining proprietary file formats again (sigh) even though I still catalog them in Lightroom. I know I could embed the raw files within DNG, but as noted in the post, this makes the files considerably larger, and it also makes workflow very janky.

I recall reading recently that the very latest version of the DNG spec does support storing manufacturer-specific opcodes, but I don't think USING them is widely supported yet in DNG-savvy software. (If I'm wrong, someone please tell me and I'll happily go back to standardizing on DNG.)

Jim

J.L. Good point on the Opcodes. I've never relied or used these presets and have never missed them. I would imagine the DNG specification will expand and contract as necessary to support options/features that are added/removed by camera manufacturers. Personally I'd rather style my own images so no loss for me. I can understand your concern though.

RJS

It's worth noting that Lightroom 2 does not auto-convert (nor does it provide an option to auto-convert) to DNG images captured during it's auto-import feature.

So, if you are tethering with auto-import enabled in Lightroom, you will find you are not getting a DNG conversion.

This is something of bummer -

Jim

RJS are you speaking to tethered shooting or being able to convert on a general import of RAW files? I know for a fact you can convert files on import, but I'm not sure on the tethered shooting side of your comment.

J Heffner

My Pentax K200D camera has the option to save as DNG. I've been using it for about a year and have had no issues.

Jim

It'd be nice if Canon and/or Nikon offered DNG as a native file format. Eventually they just might... time will tell. Thanks for the comment J.

Paul Dymond

Hi there Jim,

I started using DNG files a few years ago because of the way they display in iView Media Pro - which is what I use to catalogue my stock file.

With Canon Raw files worked on in Adobe Camera Raw, because the changes are written to the XMP file Iview couldn't read it so my Raw files looked like they'd just come out of the camera - flat and boring.

By converting the CR2 files to DNG, iView is able to see the changes made and I have lovely Raw files that can be used to make web pages, attach to emails or whatever without having to make additional small jpegs etc for different uses.

I'm a convert for sure!

Jim

Thanks for your perspective Paul. It's good to hear how others have swung around to using DNG.

Christopher Cina

I used LR 2.4 last week to convert my first batch of RAW files to DNG. Is there a need to keep the RAW files once they have been converted and you have verified the conversion was successful? It sounds like you delete the RAW files once you have done this.

Secondly, is there any difference in post-processing DNG as opposed RAW files?

Good post, answered most of my questions. Thanks.

Jim

Christopher I'm a paranoid sort so I back up the RAW files to DVD. I won't be doing this for much longer as I become more confident with the format and consistently embed the RAW file in the DNG. Hard drive space is cheap so I don't mind if my DNF files are larger than the original RAW file.

cfimages

Interesting to see you've changed over. I too looked at DNG a few years ago and didn't go with it. I'm not sure if I was aware of the metadata differences in terms of writing to DNG versus having a sidecar XMP file. It might be time for me to reconsider using DNG, because trying to re-sync everything in the event of a crash would be a nightmare. I'm curious - are you converting your archived images to DNG or just new work?

Jim

As of January I've been converting all new work created. As I have time I'll likely go back and convert older files. The more pressing issue is my newer work. Thanks for the comment Craig.