# saving metadata to file



## Ian.B (Jul 26, 2015)

I was trying to remember this 'saving metadata to file' [Ctrl+S] stuff last week and I saw another roundabout question from Emily this morning so I decided to have a look at it and see what happens. Not a lot it seems and it's very slow using Windows 7 + DNG. Photo edits don't show up on the thumbnails and it takes awhile [read _l o n g_ time] to open a preview.

So the question/s for the experts is/are: should we use "save metadata to file"?....just a dumb adobe idea? Best way to use it? [I think it worked better in bridge when I was using that]. .....I have not considered PSE12 organizer as I never look in there; maybe I should :blush:


any and all info would be handy to know.  

cheers


----------



## tspear (Jul 26, 2015)

Ian,

Ask yourself why you would have the meta data written to a file.
For myself, writing the meta data to the file is an additional level of backup. In case of catalog corruption which happens to go un-noticed and is carried into the backups so I have no good current copy; I am largely screwed. In this case, if I end up going back to a backup from a month ago or even three months ago, I can just import the images since the backup was made. I may lose the develop history, but in theory I will keep most of the final states and associated meta data. 

Since I do not mind my computer backup running in the background for days at a time, it did not bother me when I first decided to do this that it took so long to run. Since then, I have enabled the automatic write of meta data to the file; and the incremental backups handle the changes just fine.

Tim


----------



## Tony Jay (Jul 26, 2015)

Ian.B said:


> I was trying to remember this 'saving metadata to file' [Ctrl+S] stuff last week and I saw another roundabout question from Emily this morning so I decided to have a look at it and see what happens. Not a lot it seems and it's very slow using Windows 7 + DNG. Photo edits don't show up on the thumbnails and it takes awhile [read _l o n g_ time] to open a preview.
> 
> So the question/s for the experts is/are: should we use "save metadata to file"?....just a dumb adobe idea? Best way to use it? [I think it worked better in bridge when I was using that]. .....I have not considered PSE12 organizer as I never look in there; maybe I should :blush:
> 
> ...


Saving metadata to file can be handy as Tim has alluded to as an extra level of back-up.
However I would not use the automatic feature but rather only do a command/ctrl-s at the end of a session so I can leave Lightroom do its thing rather than having to have my workflow interrupted after nearly every keystroke while Lightroom saves to file.

Tony Jay


----------



## Ian.B (Jul 26, 2015)

thanks Tim and Tony. Not sure why I have not worried or thought much about this in the past even though I have known about it :crazy:. I do remember reading the suggestion to turn off auto
Not thinking windows is the best viewer for DNG/saved metadata to file; thumbnails don't changes and it takes 20-25 sec to open a preview. Need to do more experimenting I guess, not that its not much of a drama. Just handy to have if there's big problem with LR sometime


----------



## Hal P Anderson (Jul 26, 2015)

Ian,

For DNG files, LR has a command that will update the file's internal preview as well as saving the metadata. (Menu: Update DNG preview and metadata) Thiis ought to update your thumbnails.


----------



## Roelof Moorlag (Jul 26, 2015)

Updating the file with the metadata info does 'touch' every file, every time you update it.
This is not only risky (every change is a risk for corruption) but even lots of extra data to back-up


----------



## Ian.B (Jul 26, 2015)

Hal P Anderson said:


> Ian,
> 
> For DNG files, LR has a command that will update the file's internal preview as well as saving the metadata. (Menu: Update DNG preview and metadata) Thiis ought to update your thumbnails.


Than you; something else to look up



Roelof Moorlag said:


> Updating the file with the metadata info does 'touch' every file, every time you update it.
> This is not only risky (every change is a risk for corruption) but even lots of extra data to back-up


Cheers; so I take it you don't save m/data to the file (??)


----------



## happygun (Jul 27, 2015)

Roelof Moorlag said:


> Updating the file with the metadata info does 'touch' every file, every time you update it.
> This is not only risky (every change is a risk for corruption) but even lots of extra data to back-up



Are we talking about proprietary raw files or dog's here? I was under the impression that saving metadata to file only updated the xmp?


----------



## Roelof Moorlag (Jul 28, 2015)

> I was under the impression that saving metadata to file only updated the xmp?


Yes it does, but in DNG files the XMP section is part of the file. The RAW photo information is not touched but the file itself is. Your operating system and/or back-up program will 'see' a changed file and wants to back it up again.

I'm only adding the information to my database (lightroom). On export the added information can come allong (when you choose that option).
In one occasion i added all meta data to all my files and that was when i migrated from Expression Media to Lightroom a few months ago. I did a file integrity test afterwards to prevent corrupt files.


----------



## johnbeardy (Jul 28, 2015)

Roelof Moorlag said:


> Yes it does, but in DNG files the XMP section is part of the file. The RAW photo information is not touched but the file itself is. Your operating system and/or back-up program will 'see' a changed file and wants to back it up again.



The way I handle this is by storing the DNGs on a drive that is not targeted by my back up software. I can therefore write metadata to these "working DNGs" or "catalogued copies" (my own pet names) as often as I want, or not, without ever triggering any further backup.

Writing XMP metadata to your pictures doesn't require you to back them up, whether the metadata is in sidecars or is written directly to a DNG. Apart from the issue of greatly increasing backup space and  bandwidth requirements if you use DNG, remember the incompleteness of the XMP in these backups, and the underlying need of the backup workflow.

First, the XMP is mainly for communication with other apps, not backup, and therefore only offers a second-class backup which fails to include up all the work you've done on your pictures. Flags, virtual copies, stacking, history steps, assignment to collections and use in books / slideshow etc - all this work is missing from the XMP backup. The Lightroom catalogue, the lrcat file, is what contains all your work and is what must be backed up.

Second, imagine the catastrophe you're trying to protect yourself against. For 100% recovery you need two things - the images and the catalogue. You just don't need the "working DNGs" or "catalogued copies" to which metadata has been written. Instead, the combination of "virgin DNGs" (backed up when they were first created) and the catalogue restores 100% of your work.

Sure, backing up "catalogued copies" after every change gives you more than 100% peace of mind, say 150%. But that extra 50% is incomplete - and all you need is 100% recovery.

/rant 

John


----------



## tspear (Jul 30, 2015)

Roelof Moorlag said:


> Updating the file with the metadata info does 'touch' every file, every time you update it.
> This is not only risky (every change is a risk for corruption) but even lots of extra data to back-up



In what way is it risky?

Tim


----------



## Jamesp (Jul 30, 2015)

tspear said:


> In what way is it risky?
> 
> Tim



I'd be interested in this as well. I write metadata to my files as part of my regular routine. I've been doing it for years with thousands of files and have never had a problem.


----------



## Roelof Moorlag (Jul 30, 2015)

> In what way is it risky?


Every time when you are making changes to files there is a risk for corruption and you can not know if this happens..
In you catalog the file is still present and the preview shows the file correct. On first sight there is nothing wrong.
Only when you open the file (i.g. in develop mode) the RAW information from the file is used to render it. Only then it will be visible if the file is corrupt. When you have thousands of files you can not be sure all the files are ok (untill you submit them all to a visual check...)

Luckily these days Lightroom offers a built in validation option for DNG files!


----------



## tspear (Jul 31, 2015)

Roelof Moorlag said:


> Every time when you are making changes to files there is a risk for corruption and you can not know if this happens..
> In you catalog the file is still present and the preview shows the file correct. On first sight there is nothing wrong.
> Only when you open the file (i.g. in develop mode) the RAW information from the file is used to render it. Only then it will be visible if the file is corrupt. When you have thousands of files you can not be sure all the files are ok (untill you submit them all to a visual check...)
> 
> Luckily these days Lightroom offers a built in validation option for DNG files!



Actually Adobe use ACID database principles when updating the files (based on looking at the I/O rates, plus system I/O calls, I am willing to bet that Adobe actually copies the file and updates the metadata as it copies - only then does it swap the files. This is part of the reason I think the I/O hit is so bad); plus the journal system on file system is also fairly robust. 
At this point, I am more likely to trust the thousands of files with a few writes each then the catalog with thousands of writes which are not ACID compliant. 

Tim


----------



## johnbeardy (Jul 31, 2015)

Roelof Moorlag said:


> Updating the file with the metadata info does 'touch' every file, every time you update it.
> This is not only risky (every change is a risk for corruption) but even lots of extra data to back-up



No, it's not more data to back up. Back up DNGs when they are first created, and routinely back up the catalogue. When something goes wrong, this means you can recover 100% of your work.

It's a matter of reviewing your backup strategy for files with embedded metadata. Once you've done this, you can freely write or not write metadata to the catalogued copies of the files as often as you want, or not.

John


----------



## Roelof Moorlag (Jul 31, 2015)

> No, it's not more data to back up. Back up DNGs when they are first created, and routinely back up the catalogue. When something goes wrong, this means you can recover 100% of your work.
> 
> It's a matter of reviewing your backup strategy for files with embedded metadata. Once you've done this, you can freely write or not write metadata to the catalogued copies of the files as often as you want, or not.



I agree that it's not necessary to backup the DNG after each 'touch' but the filesystem and backup applications wil 'see' those files as changed. Standard they wil be part of the back-up. Your strategy does work of course but only when you know what to do.


----------

