# Migrate DNG back to RAW



## thany (May 8, 2015)

In a nutshell:
How do I migrate DNG files back to RAW without losing edits?

Long story:
I have Lightroom 5 and bought a new camera, the Olympus OM-D E-M5 mark II. This isn't supported by Lightroom (although it is by Photoshop, inexplicably) and therefor I needed to convert each and every picture to DNG before being able to do anything with them in Lightroom. I converted them by embedding the original RAW.

Now that I'm updating to Lightroom 6 (because Adobe refuses to add camera support to Lightroom 5, those nasty youknowwhats) I get my camera supported. Finally. Took way too long. Now the problem arises how to migrate my existing, edited DNGs back to RAWs without losing edits.

I can convert the DNGs back to RAWs alright, but Lightroom is going to see those RAWs as new (unedited) files. There are thousands of them, so copying over all edits of all files, and all tags and metadata and whatnot, is simply not an option. Keeping DNG is also not an option, because they are horribly inefficient, doubling every file in size. I'm actually having trouble keeping them in one place because the collection is just simply getting too big.

So how do I do this? Surely, this has to be possible. It's not the first time Adobe refuses to add camera support to Lightroom, but keeps updating DNGconverter...

I've asked Adobe FOUR times about this, and they keep ignoring (only!) this question. Which makes me like them even less. I was hoping someone here has a good idea


----------



## RonBoyd (May 8, 2015)

thany said:


> In a nutshell:
> How do I migrate DNG files back to RAW without losing edits?



I don't have an answer for you and am only answering to get clarification of what you are asking. Are you asking how to divide the DNG file into two separate files -- one the Original RAW file and the other an XMP file containing your edits. (This would be the scenario if you had processed the RAW files with ACR/LR. Forgetting for the moment that some edits are stored only in the LR Catalog.)


----------



## johnbeardy (May 8, 2015)

"Keeping DNG is also not an option, because they are horribly inefficient, doubling every file in size."

This seems a clue. DNGs are not "horribly inefficient, doubling every file in size". If you choose one option, they are usually the same or smaller than the raw file. However you may have chosen another option which embeds the raw file inside the DNG, and in that case they can double in size.

Anyway, DNGConverter's Extract button gets the raw files out of the DNGs. You're now left with DNGs in LR with your adjustments/metadata, and raw files without any work. If you import the raw files into LR, you can then choose each pair of pictures and sync the settings/metadata from DNG to raw, and this can be automated using my Syncomatic plugin.

John


----------



## mcasan (May 8, 2015)

Why do you need to do this?  You have DNG files with edits. The files are large because you told LR to embed the ORF.  So of course the files are large.  The good news is disk drives are dirty cheap.  Keep the DNGs you have and don't do anymore.


----------



## thany (May 10, 2015)

I'm not really open to things like "why'd you wanna do this?", I think I made that clear. They double in size because the RAW is still in there. And I did that because at some point I can revert to the RAWs when LR supports them, cutting the required space in half again. Storage is not "dirty cheap" when dealing with a laptop that has storage soldered on the motherboard - just don't assume anything about my environment 

The biggest problem is the edits. Both in terms of metadata, and in terms of editing in the develop module. I'll have to test that Syncomatic plugin.

/edit tested it, and €9 is too expensive for what it can do. Sorry, but no-go. I'm a hobbyist, so I'm not losing money by not buying the plugin.


----------



## RonBoyd (May 10, 2015)

Again, let me get clarification. So you are concerned about storage space caused by having to keep two copies of the RAW file -- the original (that you intend to use when LR supports them) and one embedded in the DNG file. And you wish to extract from the DNG any metadata (created after conversion to DNG) into a separate file (I suggested XMP earlier but that didn't resonate, I guess). This "metadata" file primary function would be readability by LR while processing the (original) RAW file in the Develop Module. Am I getting close?


----------



## mcasan (May 10, 2015)

Thany, sorry to have upset you.  That was not in any way the intent.  I fully understand what and why you did DNGs before Adobe was ready.  Now that Adobe products can handle E-M5II, you don't have to do embedded DNG again.  

I looked for an option to extract raw files and the related edits as sidecars from edited DNG files that have embedded raw files.  I did not find a way forward for that requirement.  The only two options I could arrive at are: 

use DNG Converter to extract the raw files and reedit the images in LR as needed
stick with the edited large DNG files from the past and only do native raw (or non embedded DNG) going forward

Sorry if none of this helps.


----------



## thany (May 11, 2015)

@mcasan Sorry if I sounded blunt, but I did want to make something clear. I appreciate your options, but they're just not helping, at least for the moment.

@RonBoyd You nailed it. That is almost exactly what I'm aiming for. But as far as I understand things, some edit are only in the LR catalog, and never in the XMP sidecars, right? With that being the case, I need some way of perhaps "renaming" the files in the LR catalog to the new RAWs they'll become after converting the DNG back to their originals.

The thing I'm concerned with, is that DNG makes my entire collection double in size. All DNGs put together amounts in about 150GB. This may not sound like a lot, but when I'm forced (by various factors) to keep this entirely on a laptop with a 256GB soldered-on SSD, it can get hairy (I know I should've bought the 512GB model... me just being frugal ). The RAWs only would've taken 70-80GB, which is well within parameters of sanity, so to speak. This is the main reason I want my DNGs converted back into RAWs, and that on its turn, is why I decided to embed the RAWs.

Now, back to the XMP sidecars. I sounds promising. Can I convert DNGs to RAW+XMP without losing anything, or do I need to do something extra somewhere somehow? Obviously, have LR store all metadata in the DNGs (just hit Ctrl+S on them, right?), but then what?


----------



## johnbeardy (May 11, 2015)

thany said:


> The thing I'm concerned with, is that DNG makes my entire collection double in size.



DNG does not make your picture collection double in size - the doubling is caused by your choice of the embed option when you made the DNGs. DNGs are usually smaller than raw files.



thany said:


> Now, back to the XMP sidecars. I sounds promising. Can I convert DNGs to RAW+XMP without losing anything, or do I need to do something extra somewhere somehow? Obviously, have LR store all metadata in the DNGs (just hit Ctrl+S on them, right?), but then what?



You cannot convert DNGs to raw+xmp, but you can extract the embedded raw files. So that leaves you with DNGs which have your LR work (after Ctrl S), and raw files which have none of your work. Syncomatic is one quick way to fix that, and you can run it on groups of 10 images without paying, but there are other solutions: 

1. Do you have SQL skills - in which case you could point the LR database to the raw files. Yuck? 
2. Have you used Exiftools - it could generate xmp files from the DNGs. Techy?
3. Have you got Bridge- it could generate xmp files from the DNGs.

2 and 3 are quite similar, but let's assume you have Bridge. First do a Ctrl S in LR, then use Explorer to make all the DNGs read only. Making them read-only is a trick which will force Bridge to create sidecars, but it won't do so until you tickle Bridge into thinking it needs to write some metadata. So select all the DNGs and apply some metadata to a field you aren't using - it might just be a colour label. Bridge should now generate sidecars which will include the adjustments and IPTC that you had entered in LR.

Now you extract the raws, put them in a folder, move those xmp files into the folder, and import the folder into LR.

How does that sound?

John


----------



## johnbeardy (May 11, 2015)

Just looking into this again, I'm currently unable to force Bridge to write sidecars for read-only files. I am 99.9999% certain it used to work and am looking into it.

It's possible that someone wrote a Lightroom plugin to do so, but I'm not sure.

John


----------



## Replytoken (May 11, 2015)

thany said:


> This may not sound like a lot, but when I'm forced (by various factors) to keep this entirely on a laptop with a 256GB soldered-on SSD, it can get hairy (I know I should've bought the 512GB model... me just being frugal ). The RAWs only would've taken 70-80GB, which is well within parameters of sanity, so to speak.



Unless you do not shoot much, and/or you have some planned hardware/storages changes in the works, it sounds like you are going to be having an impending storage issue in the not too distant future. 

Good luck,

--Ken


----------



## johnbeardy (May 11, 2015)

This would be the Exiftools batch file for Windows, creating sidecars for each DNG:

"C:\Users\john\Desktop\exiftools\exiftool.exe"  -ext dng -o %%d%%f.xmp -r "C:\Users\john\Desktop\New folder" -k

After the path to Exiftools, it says "-ext dng" or limit to DNG extension, "-o %%d%%f.xmp" says output to same directory and filename with xmp extension, "-r" means recursive, and "-k" just leaves the command window open.

As always with Exiftools, use with great care and test on a small folder of test images. If you aren't confident you understand it, don't use it.

John


----------



## RonBoyd (May 11, 2015)

thany said:


> The thing I'm concerned with, is that DNG makes my entire collection double in size.



DNG doesn't cause this situation. As I said above, this is caused by your decision to keep two copies of each RAW file -- the original and the embedded. Attempting to divert blame for this situation to DNG is unproductive (as far as this conversation is concerned).


----------



## johnbeardy (May 11, 2015)

OK, I've now found that the Bridge method won't work. While I can force it to make sidecars (making the DNGs read only and then applying an adjustment), the sidecar won't simply apply to a raw file whose name matches.

The Exiftools method does work.


----------



## thany (May 12, 2015)

I have found that when hitting Ctrl+S on a real RAW file in Lightroom and it has edits, the edits will go into XMP as well. Now the questions are:

Does Lightroom store edits in DNG in the same way it does in XMP?
Is exiftool capable of transferring every possible kind of edit from DNG to a (new) XMP sidecar?
 It feels as if that would solve this issue.


----------



## gregDT (May 12, 2015)

I'm thinking that if this problem is so unimportant that it isn't worth spending the cost of as couple of latte's and a muffin then it's not important enough to bother finding a solution for. If you have just spent over a thousand euros on a camera and there is a perfect solution to your problem that costs just nine I fail to see the problem.


----------



## johnbeardy (May 12, 2015)

thany said:


> I have found that when hitting Ctrl+S on a real RAW file in Lightroom and it has edits, the edits will go into XMP as well. Now the questions are:
> 
> Does Lightroom store edits in DNG in the same way it does in XMP?
> Is exiftool capable of transferring every possible kind of edit from DNG to a (new) XMP sidecar?



re 1, yes

re 2, probably. I didn't notice a difference, but you can test it yourself.

Or just use Syncomatic ten pictures at a time.

John


----------



## clee01l (May 12, 2015)

thany said:


> I have found that when hitting Ctrl+S on a real RAW file in Lightroom and it has edits, the edits will go into XMP as well. Now the questions are:
> 
> Does Lightroom store edits in DNG in the same way it does in XMP?
> Is exiftool capable of transferring every possible kind of edit from DNG to a (new) XMP sidecar?
> It feels as if that would solve this issue.


 Why do you want the LR edits stored in the DNG or XMP?  Only LR (or PSCC or a few other Adobe products arguing to understand what these instructions are supposed to do with the master image data.  All of the metadata associated with the Master image file is stored in the LR catalog. The metadata stored in the XMP section of the DNG file or a separate text based XMP file  is not ALL of the metadata associated with the master image file.

Looking back at your original issue, Why are you converting to DNG in the first place?  If you plan to keep the original RAW file format (and you should), The DNG is simply the same RAW data in another wrapper.  That means that your files storage requirements are nearly double (The DNG is slightly smaller than most proprietary RAW formats).  As you pointed out you can use the DNG wrapper to wrap a complete copy of the original RAW file as well as an additional copy of the image data block from the original proprietary RAW file format. 

The Smallest storage footprint is the Original proprietary RAW file and a LR catalog contraining ALL of the metadata.   Good data risk management also includes redundant back-ups of the LR catalog and the master image files.  Some one is going to point out that you can get a smaller storage footprint by converting the Proprietary RAW file and discarding the original.  I contend that only a fool would throw away the master original Proprietary RAW file in favor of keeping a smaller DNG


----------



## RonBoyd (May 12, 2015)

clee01l said:


> I contend that only a fool would throw away the master original Proprietary RAW file in favor of keeping a smaller DNG



+1   .


----------



## johnbeardy (May 12, 2015)

clee01l said:


> Looking back at your original issue, Why are you converting to DNG in the first place?



This was explained in the first post: "I have Lightroom 5 and bought a new camera, the Olympus OM-D E-M5 mark  II. This isn't supported by Lightroom (although it is by Photoshop,  inexplicably) and therefore I needed to convert each and every picture to  DNG before being able to do anything with them in Lightroom"


----------



## Hoggy (May 19, 2015)

clee01l said:


> I contend that only a fool would throw away the master original Proprietary RAW file in favor of keeping a smaller DNG



Color me a fool, then.  :nod:  I don't see a point in keeping the original.  All I'm concerned with is the image data for LR.  Of course, my Pentax cams natively save in DNG, so it's a no-brainer - but for my Canons..
But then again, I'm no pro (just yet) - and have a cognitive/memory disability, so learning LR and program X and program Y to deal with each camera would be too much for me.  It's already taken me 3 years to become somewhat proficient with LR alone.  I did contend with the possibility of using the manufacturer's software for a bit (a reason to keep proprietary raws) because I liked the Pentax JPG's SOOC, but then as I learned what could be done with RAW's I decided to just stick to learning one software package for all cameras and do away with JPG completely.
Anyhoo..  off topic.

As for the OP's issue, I think John's solutions would be the way to go.  Otherwise there are a couple plugins from Rob Cole that might do something for you (for free), but it seems his plugins page has been down for a few weeks.  If you're able to find a copy though, the relavent plugins might be XeMP, and possibly one called RAW+JPG (I know it says 'JPG', but it might still be workable somehow for your situation - Rob Cole's plugins are generally made to be altered a bit).  I do have copies I could zip up from my LR Plugins directory, but without the online documentation, they might be hard to use..  Although OTOH, he likely has put a ton of comments in the script itself.

It does befuddle me though, why LR doesn't have some sort of solution for this built-in - considering it already has an option to make embedded DNG's.  But to re-itterate what some other people are saying: the ONLY reason why your DNG file size was doubled was because of embedding the original raw - you're literally storing 2 copies of the data.  Without embedding the original, DNG's are usually smaller than proprietary raws..  Even for my Pentax cams which natively save DNG, the 'conversion' will make them smaller still - presumably because of more efficient lossless compression.  Also if you want to save on even _more_ space, you could have LR convert to DNG with NO preview and NO fast Load Data -- the no-preview option alone could save about 5 MB or so per file, since otherwise it may store a full-size jpg within the DNG.


----------



## gregDT (May 19, 2015)

clee01l said:


> I contend that only a fool would throw away the master original Proprietary RAW file in favor of keeping a smaller DNG



Shouldn't that be "I contend that only a fool would throw away the master original *Proprietary* RAW file in favor of keeping a smaller *open source* DNG"? 

To Thany - to be honest if you're seriously struggling why not just leave things alone. There are numerous image file formats and in the same vein as the utterly pointless iOS vs Android, Apple vs Microsoft arguments it's not that crucial which one you're using assuming they all store your image data. While it's true that Nikon have been utterly useless in offering any sort of decent RAW handling software for years I think the idea of them suddenly dropping the NEF format is slim to the point of being non existent. Likewise Adobe are totally invested in supporting photographers and not a specific file format, so the DNG isn't about to vanish either. Yes it would have been nice if we photographers could have had a universal, open source and future proof file format for all our digital images. But it seems that is not to be.

I work full time as a photographer and my library is full of NEF, ORF, PEF, DNG, TIF and JPEG files. I don't loose any sleep about what format my images are in. I can access them all and I can live with the very, very vague possibility that one or more format owner might somehow one day lock me out of my images files. I might get struck by lightening tomorrow as well 

Arguing about which format is the 'proper one to use' as if that's a universal rule for every photographer regardless of who they are or how they work is daft. So if you are having issues converting from format A to format B it might just be worth leaving well alone and stick with what you have. There's really no harm in having different formats.


----------



## NJHeart2Heart (Jun 25, 2015)

gregDT said:


> Shouldn't that be "I contend that only a fool would throw away the master original *Proprietary* RAW file in favor of keeping a smaller *open source* DNG"?
> Arguing about which format is the 'proper one to use' as if that's a universal rule for every photographer regardless of who they are or how they work is daft. So if you are having issues converting from format A to format B it might just be worth leaving well alone and stick with what you have. There's really no harm in having different formats.



Hi GregDT. The OP is not arguing about which format to use.  If you take a look at the original post, his concern is about space, and the reason he's trying to change formats is to gain space back on his limited computer disk.

Hope that clarifies things, as JohnBeardy did similarly above.


----------



## mcasan (Jun 26, 2015)

Lr 6/CC supports OM-D EM-5II.    Adoble does not got back and put newer camera support into past releases of Lr and Ps.   Nothing new about that.  Sometimes when you get a new camera it may take a few months before Adobe includes it.  Especially with something like the high res files from EM5II.

https://helpx.adobe.com/photoshop/camera-raw.html


----------



## Denis de Gannes (Jun 26, 2015)

johnbeardy said:


> This was explained in the first post: "I have Lightroom 5 and bought a new camera, the Olympus OM-D E-M5 mark  II. This isn't supported by Lightroom (although it is by Photoshop,  inexplicably) and therefore I needed to convert each and every picture to  DNG before being able to do anything with them in Lightroom"


The best advice here is to upgrade to LR 6 /ACR 9 which provides full support for the Olympus OM-D E-M5 MkII. Using the DNG "crotch" just to get support for a new camera model is only avoiding the real issue. 
If you are using Lightroom as your primary raw conversion software then there is no real or perceived need for DNG or XMP. Lightroom saves all your work in the Catalog file, you just need to upgrade to the latest version of Lightroom and *backup your catalog file constantly.   

*This recent video is interesting. https://luminous-landscape.com/single-video/lightroomcc6-17-to-dng-or-not/


----------



## thany (May 8, 2015)

In a nutshell:
How do I migrate DNG files back to RAW without losing edits?

Long story:
I have Lightroom 5 and bought a new camera, the Olympus OM-D E-M5 mark II. This isn't supported by Lightroom (although it is by Photoshop, inexplicably) and therefor I needed to convert each and every picture to DNG before being able to do anything with them in Lightroom. I converted them by embedding the original RAW.

Now that I'm updating to Lightroom 6 (because Adobe refuses to add camera support to Lightroom 5, those nasty youknowwhats) I get my camera supported. Finally. Took way too long. Now the problem arises how to migrate my existing, edited DNGs back to RAWs without losing edits.

I can convert the DNGs back to RAWs alright, but Lightroom is going to see those RAWs as new (unedited) files. There are thousands of them, so copying over all edits of all files, and all tags and metadata and whatnot, is simply not an option. Keeping DNG is also not an option, because they are horribly inefficient, doubling every file in size. I'm actually having trouble keeping them in one place because the collection is just simply getting too big.

So how do I do this? Surely, this has to be possible. It's not the first time Adobe refuses to add camera support to Lightroom, but keeps updating DNGconverter...

I've asked Adobe FOUR times about this, and they keep ignoring (only!) this question. Which makes me like them even less. I was hoping someone here has a good idea


----------



## johnbeardy (Jun 26, 2015)

Denis de Gannes said:


> The best advice here is to upgrade to LR 6  /ACR 9 which provides full support for the Olympus OM-D E-M5 MkII.  Using the DNG "crotch" just to get support for a new camera model is  only avoiding the real issue.



I think you meant to say crutch, not crotch 

There are benefits from using DNG but that's not what this old thread is about.


----------



## thany (Jun 29, 2015)

mcasan said:


> Lr 6/CC supports OM-D EM-5II.    Adoble does not got back and put newer camera support into past releases of Lr and Ps.   Nothing new about that.  Sometimes when you get a new camera it may take a few months before Adobe includes it.  Especially with something like the high res files from EM5II.
> 
> https://helpx.adobe.com/photoshop/camera-raw.html


But here's the thing. My camera came out BEFORE LR6 was released, or even announced. I had no way of knowing Adobe was going to screw me over (sorry for blunt language) and basically force me to pay for my camera getting support (and those other huge bugs that exist in LR5 and may possiblymaybe be fixed in LR6).

I'd love to upgrade to LR6, but I don't exactly feel an urge to go and pay for it, now that Adobe has pulled this. But still, after having done so, I'd still like to backport my DNG files to RAW, since currently they take up about double the space. I have more than enough (offsite) storage, ut on the go in my laptop, I'm limited. Also, it's a waste to storage things twice. That's the whole reason I want to backport DNG to RAW.


----------



## Anthony.Ralph (Jun 29, 2015)

As I read it, the OP is concerned with the large size of his current DNG files and the effect on limited storage capacity. I *think* I have a partial solution - feel free to point out my errors if you see them.

1. Extract a group of RAW files from their DNG wrapper.
2. Delete the DNGs
3. Reconvert the RAW files to DNG - but don't select the 'include raw' inside option.
$. Delete the RAW files.

The result should be less space taken up and, assuming the resulting file names are the same as the RAW+DNG version, Lightroom will use any existing edits to these 'new' DNGs quite happily.

I tried it on a single file as 'proof of concept' and it looked okay to me. Caveats: try a test first, always have back up copies, don't blame me if there is any problem with this 

Anthony.


----------



## tspear (Jun 29, 2015)

thany said:


> But here's the thing. My camera came out BEFORE LR6 was released, or even announced. I had no way of knowing Adobe was going to screw me over (sorry for blunt language) and basically force me to pay for my camera getting support (and those other huge bugs that exist in LR5 and may possiblymaybe be fixed in LR6).
> 
> I'd love to upgrade to LR6, but I don't exactly feel an urge to go and pay for it, now that Adobe has pulled this. But still, after having done so, I'd still like to backport my DNG files to RAW, since currently they take up about double the space. I have more than enough (offsite) storage, ut on the go in my laptop, I'm limited. Also, it's a waste to storage things twice. That's the whole reason I want to backport DNG to RAW.



Sorry, but I find this kinda funny. Bug fixes for a reasonable period, and that period can be debated, should be expected; more due to the fact that software is no longer adequately tested and has become ever more complicated before it is released.
But why do you expect Adobe to provide a free upgrade? Supporting your camera is a new feature.

Tim


----------



## Jim Wilde (Jun 29, 2015)

tspear said:


> But why do you expect Adobe to provide a free upgrade? Supporting your camera is a new feature.



Yes and no, Tim. New camera support is typically added (for free) in the "dot releases" where possible, but there will always be the situation where support for new models that don't make it into (what turns out to be) the last dot release of a particular LR version will fall into the next LR version (and thus not free). 

It's not a case of "Adobe screwing you over", it's just a case of unfortunate timing. Happens at the end of each LR version.


----------



## tspear (Jun 29, 2015)

Jim,

I agree. 

Tim


----------



## thany (Jun 30, 2015)

Jim Wilde said:


> Yes and no, Tim. New camera support is typically added (for free) in the "dot releases" where possible, but there will always be the situation where support for new models that don't make it into (what turns out to be) the last dot release of a particular LR version will fall into the next LR version (and thus not free).
> 
> It's not a case of "Adobe screwing you over", it's just a case of unfortunate timing. Happens at the end of each LR version.



In my defense, I had no way of knowing LR6 was around the corner. It wasn't announced yet. For all I knew, LR 5.8 may have come out in the mean time.
Secondly, I don't see why Adobe blatantly refuses to put new camera support in existing versions of LR. After all, Photoshop CS6, which is much older, does get new camera support. On top of that, Adobe should realize that not everyone upgrades to the newest versions of their software all the time. On top of that yet again, the E-M5 Mk.II is ridiculously simple to support - its raws are virtually (if not completely) identical to the Mk.I raws.

Having said that, this thread isn't actually about that. This thread is about the best/easiest/fastest way to backport DNG's-with-embedded-RAW back to their original RAWs while keeping metadata and edits and all that.


----------



## Jim Wilde (Jun 30, 2015)

thany said:


> I don't see why Adobe blatantly refuses to put new camera support in existing versions of LR.



Seriously? You think Adobe should continue to provide updates for new cameras to LR1, LR2, LR3, LR4, LR5 as well as LR6? 

That's totally impractical (your PS CS6 example is only possible because it's actually the ACR plug-in that gets updated), and is one of the reasons why Adobe has made available the free DNG converter so that the latest cameras *are* supported as far back as LR1.


----------



## johnbeardy (Jun 30, 2015)

thany said:


> Secondly, I don't see why Adobe blatantly refuses to put new camera support in existing versions of LR.



And I don't see any excuse for camera makers failing to offer DNG as an option, just as they offer JPEG. In other words, the blame is misplaced.


----------



## Anthony.Ralph (Jun 30, 2015)

I thought the OP was concerned with dealing with a size/capacity problem on his SSD using DNG with the RAW inside the wrapper. It would seem he prefers to have a grumble about Adobe and their support for old versions of Lightroom. Is he still interested in dealing with the original problem or not?

Anthony.


----------



## rob211 (Jul 1, 2015)

Anthony.Ralph said:


> I thought the OP was concerned with dealing with a size/capacity problem on his SSD using DNG with the RAW inside the wrapper. It would seem he prefers to have a grumble about Adobe and their support for old versions of Lightroom. Is he still interested in dealing with the original problem or not?
> 
> Anthony.


Apparently he did upgrade and format is supported.

I think the easiest way to get space saving is this:

Open the DNG with the embedded RAW in Adobe's DNG Converter application.

Choose conversion options, which would include NOT embedding the RAW. You'd have to add a "1" or something to the file name so as to not overwrite, to be on the safe side.

Convert, and then reimport into Lr. It would keep keywords, etc etc. Then you could batch rename, delete the embedded RAW-DNGs, etc. I think it retains all the same info.

Yes, you don't end up with an .orf but you do have the info. You really haven't lost anything but space. And then going forward just use the .orfs instead.


----------



## mcasan (Jul 3, 2015)

The reasons for the initial non-support of E-M5II was it has a high res mode that takes several raw files and merges them into 1 large high resolution raw file.  When the camera was released, LR was not ready for the high res mode.    This is potentially part of the pain of purchasing a bleeding edge camera body that does something special.

Olympus will likely introduce E-M10II later this year and may do something like hi res mode that could need updates in post processing software.   No doubt that will happen again next September when Olympus introduces E-M1II with all sorts of new bells and whistles.


----------



## rob211 (Jul 4, 2015)

Yeah, every camera that does proprietary RAW as opposed to DNG output has this problem. You'd think they'd give Adobe and others a head start on RAW support, but I don't know if they even cooperate. Anyone know?

At least Adobe gets on it pretty quick; the open source RAW support is often even faster. And that's way better than Apple, which took like 6+ months for my last Oly camera.


----------



## mcasan (Jul 10, 2015)

Given that special circumstances the import and conversion to DNG was OK.   The problem is not keeping a copy of the original ORF raw files that could now be imported in native format.   A circumstance like this is about the only way I would import to DNG.  But then I would keep a copy of the native format until LR could handle it.


----------



## pknight (Jul 30, 2015)

A bit tangential, but since the issue has been raised in this thread...One reason why non-fools might convert native RAW to DNG is that some software programs drop support for older RAW formats in more recent versions.  This means that if you wanted to use the latest and greatest version of such software with files from your old, now unsupported, camera, as well as your new, supported camera, you would be out of luck.  You would have to have multiple versions of the software, with different features and capabilities, in order to edit all of your files.  Unless  you  had converted to DNG...

Right now, the current version of Canon's DPP does not support files from three of the five Canon bodies I have owned (350D, 30D, 40D), among many others.  Does anyone feel completely confident that Adobe is incapable of making such a decision?  I would not be in the least surprised if Adobe dropped support for older cameras at some time.  I hope they don't, but then I wish they hadn't made a number of decisions that they have recently made.

I also don't quite understand the aversion to DNG files.  Other than the time it takes to convert the original RAW format to DNG, there is no disadvantage to DNG compared to the original format.  DNGs contain all of the data present in the original format, and behave exactly like the originals in editing.  Once converted to DNG to obtain software compatibility, what is the purpose of hanging on to the original?


----------



## tspear (Jul 31, 2015)

pknight,

Actually I have read it stated by multiple who know a lot more than I, that there is a loss of some data from raw to DNG. That is why there is an option to embed the raw file in the DNG. Now how much of that lost data is important is for now an academic debate because no software can take advantage of the additional data. So the theory is that one day someone may discover how to recover some additional detail of information from that lost data. And therefore the raw file is better.

I personally think that if someone gleans a magical method to get more information from the raw file, it will be done with a newer file format anyway, not one of the older formats; and Adobe will quickly follow suite; and I will have to use bridge to "re-convert" some files again. shrug. it may happen; but I bet on data corruption caused by SW/HW happening first. So DNGs help me protect against that.

Tim


----------



## pknight (Jul 31, 2015)

I would be very pleased if a reference to an Adobe source (having developed and promoted DNG, they would know) outlining the loss of data in DNG could be provided.  Official descriptions of DNG describe the process as lossless.  As an open specification, parties other than Adobe develop DNG converters, and I would not be alarmed to learn that one or more of those programs screws things up.  As far as I can tell, claims of DNG data loss are primarily found on user forums, with the associated reliability of those claims.

However, I suspect that the perception of lost data comes from DNG's use of lossless compression, and the stripping of some proprietary metadata that is used by the camera manufacturer's RAW converter and editing software, and not needed in, nor compatible with, a cross-platform format.  There is no loss of image data, at least in Adobe's conversion.

In Adobe's specification document for DNG they briefly list the advantages, as they see them, of DNG.  Of these, one resonates with me, and that is camera manufacturers who drop support for older RAW formats in their software.  Canon has done exactly this, and you will need multiple versions of DPP to edit all of your files if you have images from both older and newer Canon cameras, as I do.  If Canon can drop support for its own cameras, it would be no surprise if, some day, ACR no longer supports old file formats, and DNG will sure look good then.


----------



## tspear (Jul 31, 2015)

pknight,

I was referencing the metadata. One of the suggested examples I have previously seen is with Nikon files. Nikon image data is stored in a specific format, and includes additional sensor data not part of the image but with the camera environment. The argument is that some genius in the future using this meta data will be able to infer/deduce some additional information about the image which is dependent on the raw sensor data being in the native order.

As I said before, I think if this ever occurs it will be applied to the newer formats and Adobe would add it. 

Tim


----------



## pknight (Jul 31, 2015)

Thanks.  I wonder if Nikon's own software utilizes this data?  It might be difficult to incorporate this sort of information from many camera manufacturers into DNG, since each company will have unique information stored, perhaps, in unique ways.


----------



## tspear (Jul 31, 2015)

From the comments and speculation I have read. No one yet has leveraged the meta-data yet.
But that has been the persistent theory that someday someone will. I will cross that bridge when it comes....

Tim


----------



## rob211 (Jul 31, 2015)

I'd also like to see what metadata is lost. I'm very skeptical. I've seen folks run diffs on the metadata, and there ARE differences. But they do NOT relate to underlying image data, the raw of the RAW if you will. Some relates to conforming incorrect Nikon use of exif tags. Some relates to conforming to standards (lens info, for example; DNG will add the lens number to the tag). A lot relates to the preview; the nef will contain a Nikon-generated preview; a DNG will have a different preview (perhaps you can retain the camera preview, I dunno). So that's different.

The RAW's metadata might contain something peculiar to the settings of the camera that don't affect the image; I have some RAWs that contain the camera's battery voltage. No need to keep that. Someone mentioned in a post somewhere that an "affinetune" value was lost. Probably; since it relates to how the camera takes the picture, not the result. Unlike other stuff, you can't change the focus in post. But perhaps DNG now includes it; I dunno. You can see a nerd-tacular diff breakdown of a nef vs dng here, amusing even if you don't own a Nikon: http://monochrome.sutic.nu/2014/12/17/nef-to-dng-conversion.html

There are some very esoteric differences in some stuff, like perhaps edge pixels with false values for calibration purposes or somesuch. But for any realistic purpose the image data is the image data.

Finally, no one is going to come up with anything to bring out some hidden worth in a RAW that isn't in the corresponding DNG. Hasn't happened yet. And there's no incentive for doing so in the future. Even if a teensy bit of extra info could be coaxed out in five years from the nef vs the dng, that info will still be insignificant since it will pale in comparison to new software tools that adjust and convert the main image data.


----------



## clee01l (Aug 1, 2015)

pknight said:


> Thanks.  I wonder if Nikon's own software utilizes this data?  It might be difficult to incorporate this sort of information from many camera manufacturers into DNG, since each company will have unique information stored, perhaps, in unique ways.


Actually what you are referencing is a Metadata field called "Manufacturer's Notes"  It is an EXIF undefined structure and the manufacturer can structure it any way that suits.  Not only does Nikon use this but so does Canon and I suppose others as well.  Canon Stores the data for ALO & HTP and Nikon stores the ADL data there. Of course the proprietary structures are not compatible and Adobe chooses to ignore that data. Even the recommendations from Nikon Professionals say to turn off ADL if you are Post processing RAW with even the Nikon apps.  ALO, HTP and ADL are still available in the DNG header, but the only apps that make use of this data is proprietary and only works with a Proprietary RAW file format.


----------



## pknight (Aug 1, 2015)

Well, it sounds to me that whatever info is being ignored by DNG conversion is unrelated to the image and its quality, which leaves the lossless compression, and up to 20% space savings according to some sources, as the most meaningful change made during conversion.  This is a good thing, not a problem.  Combined with the demonstrated willingness of camera manufacturers to abandon old RAW formats, I fail to see any serious downside to DNG, aside from the time it takes for the conversion.


----------



## Hoggy (Aug 1, 2015)

pknight said:


> .... which leaves the lossless compression, and up to 20% space savings according to some sources, as the most meaningful change made during conversion. .... I fail to see any serious downside to DNG, aside from the time it takes for the conversion.



Don't forget the biggest meaningful change of them all:  added verification checksums of the image data itself.  I even converted my old jpg collection to DNG due to that.  And it's already saved me several files.


----------



## Jknights (Sep 6, 2015)

johnbeardy said:


> DNG does not make your picture collection double in size - the doubling is caused by your choice of the embed option when you made the DNGs. DNGs are usually smaller than raw files.
> 
> 
> 
> ...




Superb solution John.
This really is the way to go if you have used DNG and want to extract the xmp and the RAW files.


I would always recommend that they keep the original RAW files separate even if you want to make DNGs.  That way you have the best of both worlds. You can make lite DNGs and you have the original RAW for alternative processing softwares.
I have 125000+ images all RAW from the last 10 years and it is on one 4TB disk with space for more!   (Yes it is backed up x2 - Dont call me paranoid!).


----------



## thany (May 8, 2015)

In a nutshell:
How do I migrate DNG files back to RAW without losing edits?

Long story:
I have Lightroom 5 and bought a new camera, the Olympus OM-D E-M5 mark II. This isn't supported by Lightroom (although it is by Photoshop, inexplicably) and therefor I needed to convert each and every picture to DNG before being able to do anything with them in Lightroom. I converted them by embedding the original RAW.

Now that I'm updating to Lightroom 6 (because Adobe refuses to add camera support to Lightroom 5, those nasty youknowwhats) I get my camera supported. Finally. Took way too long. Now the problem arises how to migrate my existing, edited DNGs back to RAWs without losing edits.

I can convert the DNGs back to RAWs alright, but Lightroom is going to see those RAWs as new (unedited) files. There are thousands of them, so copying over all edits of all files, and all tags and metadata and whatnot, is simply not an option. Keeping DNG is also not an option, because they are horribly inefficient, doubling every file in size. I'm actually having trouble keeping them in one place because the collection is just simply getting too big.

So how do I do this? Surely, this has to be possible. It's not the first time Adobe refuses to add camera support to Lightroom, but keeps updating DNGconverter...

I've asked Adobe FOUR times about this, and they keep ignoring (only!) this question. Which makes me like them even less. I was hoping someone here has a good idea


----------



## Jknights (Sep 6, 2015)

pknight said:


> Well, it sounds to me that whatever info is being ignored by DNG conversion is unrelated to the image and its quality, which leaves the lossless compression, and up to 20% space savings according to some sources, as the most meaningful change made during conversion.  This is a good thing, not a problem.  Combined with the demonstrated willingness of camera manufacturers to abandon old RAW formats, I fail to see any serious downside to DNG, aside from the time it takes for the conversion.



People are discussing this with the belief that Adobe never lie or die.
I do not trust Adobe or any other large corporate that is run for profit!.  I am not against profit but against the ethics and moralities that emerge from corporations run by/for accountants and shareholders.
If you believe that Adobe will always be around and treat you fairly, then we need to just agree to differ!  
Case in point is the subscription v. outright purchase issue with Creative Collection.   I am a long term user of Adobe products since 1995!  I will not buy subscription products no matter how good it seems for other people, it does not work for me.


I do however totally agree with you that this could all be avoided by the camera manufacturers being more open about (publishing) their RAW format.  Also I think that the camera manufacturers should give the camera users the option to save images in RAW, Adobe DNG, TIFF and JPG formats.  If this came about then Adobe DNG would be more acceptable to me.  
I do applaud Adobe for developing the DNG format specification as well as making it open for use and exploitation in both their softwares and other developers.

I think that there are vested interests in play and until there is give by the camera manufacturers then Adobe get somewhat caught in the middle between the end user and the camera manufacturer.
The whole issue about how 'badly' Adobe ACR and LR handle the Fuji X sensor RAW files is a case in point.  Personally I can see the issue in a very small proportion of my images.   These images can be processed and yield better results in other softwares but my LR and ACR results are perfectly good for 99% of my images so for me it is a non-issue.


----------



## clee01l (Sep 6, 2015)

Jknights said:


> Also I think that the camera manufacturers should give the camera users the option to save images in RAW, Adobe DNG, TIFF and JPG formats.  If this came about then Adobe DNG would be more acceptable to me.
> I do applaud Adobe for developing the DNG format specification ...


There are at least 12 camera manufactures that to offer DNG as a RAW format option. (Pentax & Leica are prominent names).   Adobe owns the TIFF standard as well as the DNG.  DNG is a subset of the TIFF/EP6 standard. So, for a fact is the proprietary NEF and CR2 RAW formats from Nikon and Canon and possibly others. The problem with proprietary RAW formats like NEF & CR2 is not that they are not well documented, but that they are not extensible like the DNG RAW format.


----------



## Denis de Gannes (Sep 7, 2015)

clee01l said:


> There are at least 12 camera manufactures that to offer DNG as a RAW format option. (Pentax & Leica are prominent names).   Adobe owns the TIFF standard as well as the DNG.  DNG is a subset of the TIFF/EP6 standard. So, for a fact is the proprietary NEF and CR2 RAW formats from Nikon and Canon and possibly others. The problem with proprietary RAW formats like NEF & CR2 is not that they are not well documented, but that they are not extensible like the DNG RAW format.


At the present time I do not feel threatened that my cameras files will be abandoned in the short term. My earliest raw files are from my Olympus E20n which I purchased in 2002 and the files are still supported by ACR9.1.1/LR 6.1.1, and Capture One Pro, SilkyPix, Qimage Ultimate, and Olympus Viewer 3. and I am sure by several other raw conversion software. Actually its not supported by DXO as they were not even around in 2002; they also do not support the files from my Olympus E300 which I purchased in 2005. There is a life without DNG.
As a result I do not DNG or XMP for my work with LR which I have adopted since inception in 2006 but am quite satisfied with my backups of the Catalog file.


----------



## Hoggy (Sep 7, 2015)

True that even the oldest of the proprietary raws will likely still be supported by many...  But good luck with easily detecting corruption of the image data in those.  :razz:


----------



## rob211 (Sep 7, 2015)

Jknights said:


> People are discussing this with the belief that Adobe never lie or die.
> I do not trust Adobe or any other large corporate that is run for profit!.  I am not against profit but against the ethics and moralities that emerge from corporations run by/for accountants and shareholders.
> If you believe that Adobe will always be around and treat you fairly, then we need to just agree to differ!
> Case in point is the subscription v. outright purchase issue with Creative Collection.   I am a long term user of Adobe products since 1995!  I will not buy subscription products no matter how good it seems for other people, it does not work for me.
> ...


Well, even if you buy equipment made by the People's Republic of Photostan you've got the same issue of longevity. As Cletus noted, however, Adobe owns some standards and PSDs have become almost a de facto standard. And even if Adobe went up in a puff of smoke, other developer could easily process DNGs; they don't require the same reverse engineering that RAW files from manufacturers do.

If you like paranoid visions to consider, then take proprietary RAW producers (Fuji, but also Canon, Oly, Panny, etc) as examples. You're right about Adobe (and Apple, Phase One, etc etc) being caught in the middle. Manufacturers could make it much easier for RAW developers, but they don't. It's a complete mystery to me. Do they really think the people who spend big money on cameras are using the crapware they bundled on DVDs with their photos? Are they afraid that other camera manufacturers will figure out the computer processing within their cameras? If so, here's the paranoid fantasy: that they'll make it impossible to reverse engineer those RAWs. Or so difficult no one bothers to do so.

A smart digital camera buyer considers what software is required to process images BEFORE purchasing the camera hardware. I most recently went with Pentax because I'm fed up with crippled file formats that the other camera manufacturers use. Not everyone can make that choice, but maybe if we did it more often they'd wise up. The evils of Adobe are far down my list behind the evils of the camera makers. Especially since they aren't exactly experienced unprecedented sales.


----------



## Hoggy (Sep 7, 2015)

rob211 said:


> You're right about Adobe (and Apple, Phase One, etc etc) being caught in the middle. Manufacturers could make it much easier for RAW developers, but they don't. It's a complete mystery to me. Do they really think the people who spend big money on cameras are using the crapware they bundled on DVDs with their photos?



For real about the crapware bundled with the manufacturer software!  And speculation is that some manufacturers will likely never open up their formats.  For now it seems a pipe dream that the sticklers will ever do so.  And like you, I just can't fathom why they want to keep their formats so closed..??



> A smart digital camera buyer considers what software is required to process images BEFORE purchasing the camera hardware. I most recently went with Pentax because I'm fed up with crippled file formats that the other camera manufacturers use.



Good choice on the Pentax.   Not only the open DNG format, but one can also use any lens ever made for the K-mount, unlike others which seem to periodically change mounts.   If Pentax also made super-compacts that record raw like Canon/Sony, I'd get it in a heartbeat.  ..But I digress..  Don't want to make this thread a manufacturers war..



> Not everyone can make that choice, but maybe if we did it more often they'd wise up. The evils of Adobe are far down my list behind the evils of the camera makers.



Exactly.


----------

