# Gah — used copy of Catalog & Catalog over three days.  How to merge/recover?



## Kirby Krieger (Oct 18, 2018)

I screwed up.  I moved a large Catalog to a new drive, launched Lightroom by double-clicking the Catalog file on that drive, and assumed that LR would now default to this Catalog.  Which is does when you launch Lightroom via the Lightroom program.  I overlooked that I launch Lightroom from LaunchBar, and that that shortcut calls not the Lightroom program but rather calls the Catalog file — the now old, retired one.

So I have edits from the last three days in each Catalog.  Furthermore, in trouble-shooting I turned "Write XMP to files" back on.

I  have accounted for the different number of Photos in the two Catalogs (11 Photos in total).  I can re-create the work I remember doing, but I did some tagging and other database work that I won't recall (and I don't know without looking which Catalog holds which changes).  Since I can open only one Catalog at a time, it is cumbersome to compare the two.

The total number of Photos I've edited in the last five days is about 500, which includes about 400 Photos that are almost all simply imported.  I have about 100 Photos that have changes to metadata and edits post-import.  There are probably two dozen Photos that have been edited in PSCC.

Is there any good way — or barring that, a rational way — to proceed?  My goal is to create one up-to-date Catalog on the new drive.  I will ensure that I don't use the old Catalog(s) (I'd saved it as a backup) by renaming them (which I should have done right away).

TY.


----------



## Johan Elzenga (Oct 18, 2018)

Choose one of the catalogs as the main catalog, then choose ‘Import from Another Catalog’ and import the other.


----------



## Kirby Krieger (Oct 18, 2018)

JohanElzenga said:


> Choose one of the catalogs as the main catalog, then choose ‘Import from Another Catalog’ and import the other.



Thanks for the speedy help   .  Does it make any difference which I chose as the main one?  IOW, will the resulting "main" Catalog be the same regardless of which of the two existing Catalogs I select?

{Added:}
How will Lightroom handle a Photo that has been edited in each Catalog since I copied the Catalog?


----------



## johnbeardy (Oct 18, 2018)

It should give priority to the more-recently edited image. At this point, I would take a backup of each catalogue, just in case.


----------



## Johan Elzenga (Oct 18, 2018)

Kirby Krieger said:


> How will Lightroom handle a Photo that has been edited in each Catalog since I copied the Catalog?


Lightroom will ask you what you want to do. Ignore, keep the most recent edit, or create a virtual copy so you have both versions.


----------



## Kirby Krieger (Oct 18, 2018)

JohanElzenga said:


> Lightroom will ask you what you want to do. Ignore, keep the most recent edit, or create a virtual copy so you have both versions.


Perfect — re-assuring news.  Thanks  .  
Kudos to Adobe for making this thorough, exact, and easy.
Been running the import for a couple hours.  I expected it to take a long time.  Still "Checking for changed duplicate photos".


----------



## Kirby Krieger (Oct 18, 2018)

JohanElzenga said:


> Lightroom will ask you what you want to do. Ignore, keep the most recent edit, or create a virtual copy so you have both versions.


Alas.  The dialog that popped up isn't entirely clear.  And I have an additional issue.

I would _love_ to simply keep the most recent edit.  I don't see how I can do that.  The options listed for handling the "Changed Existing Photos" are:
- Replace Nothing
- Replace Metadata and develop settings only {and}
- Replace Metadata, develop settings, and negative files

The two "Replace" options activate a checkbox to "Preserve old settings as a virtual copy".

It's not clear to me that Lightroom is aware of, and will act on, which of any pair of conflicting Photos will be preserved/used as the "original".  You say that Lightroom will keep the most recently edited Photo (or set it as the "original") but I see nothing in the dialog that indicates such.  The first choice (other than do nothing) is simply "Replace" — there is no indication that it will replace _only_ those Photos with __older__ metadata and edits.  The second choice's option says "preserve _old_ settings, not "preserve _older_ settings" — perhaps a minor distinction, but one which fits with the rest of the indications in the dialog.  When I hover the mouse cursor over one of the thumbnail previews of the Photos for which there are conflicts in the metadata and/or edits between the two Catalogs, a pop-up informs me "The settings from the imported file will replace the settings for this existing file" — at least for all the thumbnails I've hovered over so far.  It appears to me that the selected Photos are simply the ones in the to-be-imported Catalog that have metadata and/or edits that are different from the Photos in the being-imported-into Catalog, from which I conclude that no distinction based on _the time-stamp_ of the change is being made.  Lightroom seems to neither distinguish nor care which metadata and/edits are more recent.

The choices presented thus seem to be:
- do nothing
- replace the metadata and/or edits for _every_ photo that shows a conflict, {and}
- replace the metadata and/or edits for _every_ photo that shows a conflict and create a new Virtual Copy with the metadata and/or edits present in the Photo in the being-imported-into Catalog.

That's not helpful.  (Using the _most-recent_ change would be very helpful.)

Am I mis-reading the dialog?

I'm showing 28,000 Photos with conflicting metadata and/or edits.  It would also be quite useful to be able to know the differences in the metadata and/or edits.  There's no way I made 28,000 changes that are important to me.  I suspect nearly all of these came from when I turned on "Write XML to files" before I'd thought it through.  I have no idea what the conflicts are. 

I'm tempted to create a new small Catalog by exporting the 100 Photos I'm worried about from one of the Catalogs and importing them into the other.  I can process one hundred pairs manually.  But that leaves me deciding which Catalog to use as the new "main" Catalog — and what to do about the other 27,900 conflicts in the metadata and/or edits.

Thanks for sharing your expertise.

{Added:]
I was able to get a slide-down panel to show.  This, too, supports my interpretation (that Lightroom is not making a distinction based on the time-stamp of the change to the metadata and/or edits).


----------



## Kirby Krieger (Oct 19, 2018)

I've just looked this up in my copy of The Lightroom Queen's book (I have The Lightroom CC/6 Missing FAQ, with hundreds of annotations).  Merging Catalogs is covered in "Chapter 21 - Multiple Computers Or Catalogs".   Bampton makes it clear that the settings in the source Catalog over-write the settings in the target Catalog when Catalogs are merged.  There is one line — it's at the top of p. 503 under the sub-heading "Cons" — where this is mis-stated.  She writes, "_You must keep track of who is working on each chunk of photos, as the latest edits win when merging catalogs._"  In fact, the latest edit does not win.  Lightroom makes no attempt to determine which is the latest edit (according to the rest of the book and my own experience).  The edit in the source Catalog always "wins".

I think — posters above expected this was already the case — that the user should be given the chance to have Lightroom assign _the more-recent settings_ to the "original"/master, and optionally assign the less-recent settings to a newly-created Virtual Copy.

My current plan is make the back-up of my Catalog made most-recently before I copied the Catalog into a new "main" Catalog (this should eliminate my problem with the 27,500 variant settings caused by my writing XMP back to the files) and then import into that Catalog sets of Photos edited in the last 3 days from each of the current duplicate Catalogs, and with each import make Virtual Copies.  Then I will manually deal with each of the 500 Photos edited or importing since I copied the Catalog.  About 400 of these shouldn't have any Virtual Copies — they are untouched since import.  The last 100 Photos will have one or two Virtual Copies.  I will deal with each of them individually.

How's that sound?


----------



## johnbeardy (Oct 19, 2018)

I tested this "latest edits wins" long ago, which is why I suggested it. OK, it could have broken or changed, but I doubt it. In any case you could test it with two small catalogues, but if you have solid backups there's no harm in pushing ahead and seeing if Import from Catalog works as it should.

It does make sense to import only a small catalogue with the recent edits.


----------



## Johan Elzenga (Oct 19, 2018)

I would use the virtual copy option. Save both edits so you can check yourself which ones you want to keep.


----------



## johnbeardy (Oct 19, 2018)

Why do you think resolving the problem manually is a good idea - when the automated system does work?

OK, if it's the case that a photo might have been edited in each catalogue, "newer wins" might not be possible in every or in enough cases. Then one might choose the VC method and resolve clashes manually. My Syncomatic can speed things up if it is possible to say "in this folder I want the VCs to update the masters" - you could update 10 masters at a time without buying.


----------



## Johan Elzenga (Oct 19, 2018)

Yes, that was my idea. The newest edit is not necessarily the best one if edits were done in both catalogs. And it looks like that 'newest wins' does not always work as expected, as suggested by that error dialog.


----------



## johnbeardy (Oct 19, 2018)

I've never seen that dialog box, but it's conceivable that it's more recent than my testing.


----------



## Kirby Krieger (Oct 19, 2018)

Thank you John @johnbeardy and Johan @JohanElzenga a for your helpful responses.  I will attempt to do some definitive testing today to determine how conflicts in the settings ( = metadata and/or edits) of Photos common to two Catalogs being merged are handled.  I will respond to the other questions, suggestions, etc. later today.  Thanks again for helping me resolve this issue well.

—Kirby.


----------



## Kirby Krieger (Oct 19, 2018)

johnbeardy said:


> I tested this "latest edits wins" long ago, which is why I suggested it. OK, it could have broken or changed, but I doubt it. In any case you could test it with two small catalogues, but if you have solid backups there's no harm in pushing ahead and seeing if Import from Catalog works as it should.
> 
> It does make sense to import only a small catalogue with the recent edits.


I tested it.  This reply addresses the current behavior of Lightroom when it merges Catalogs.

Lightroom does not compare the date-stamp of the settings of the Photos when it selects between Photos with conflicting settings.  Lightroom will always use the Photo in the source Catalog as the “master” Photo.  If you check “Preserve old settings as virtual copy”, Lightroom will make the Photo from the source Catalog the “master” Photo and will create a Virtual Copy with the settings of the Photo in the target Catalog.

IOW, when merging Catalogs, as long as you agree to change existing Photos, the Photos in the source Catalog will over-write the Photos in the target Catalog, regardless of when either Photo was last changed.

This was somewhat straightforward to test.  I created and populated a new Catalog, I made some changes in both metadata and development settings, I closed and copied the catalog, I opened each Catalog in turn and made some changes in both metadata and development settings, keeping track of which Photos had been changed before the second Catalog was made, which changes had been made in the second Catalog, and which changes had been made in the first Catalog after the the second one had been made, and which of those changes were on Photos that had already been changed.  Then I made two additional copies of the first Catalog, and imported the second Catalog into each one, once with “Preserve old settings as a virtual copy” checked, and with it not checked.  Then it was simply a matter of looking at the third and fourth Catalogs to see what Lightroom had done.

In every case, Lightroom used the Photo in the source Catalog for the settings to the “master” Photo in the target Catalog.

In no case did Lightroom make any distinction based on the time-stamp of the edits to the Photos.

The wording of the checkbox in the dialog is misleading.  The settings that may be preserved in a Virtual Copy are not the “old settings”, they are the settings in the target Catalog.

Bampton states this on p. 513 of her Lightroom CC/6 book:
“_**Preserve old settings as a virtual copy** is useful if you’ve accidentally edited the photos in both catalogs. It creates a virtual copy of the target catalog’s settings and imports the source catalog’s settings as the master photo. Once the import completes, you can check through the photos and determine which version to keep._”

This is how Lightroom Classic CC (desktop) currently works.  What this does to each changed Photo's "metadata status" — which Lightroom tracks and which can be used as a criterion in Smart Collections — gets complicated when Catalogs are set to write XMP to disk.  I will address that in my next post.


----------



## Kirby Krieger (Oct 19, 2018)

JohanElzenga said:


> I would use the virtual copy option. Save both edits so you can check yourself which ones you want to keep.


The problem that approach presents is that there are 28,000 Photos that Lightroom identifies as having changed.  I can't go through 28,000 Photos to make the decision whether to keep or not.

(Edited for clarity.)


----------



## Kirby Krieger (Oct 19, 2018)

johnbeardy said:


> Why do you think resolving the problem manually is a good idea - when the automated system does work?
> 
> OK, if it's the case that a photo might have been edited in each catalogue, "newer wins" might not be possible in every or in enough cases. Then one might choose the VC method and resolve clashes manually.


I hope it's now clear that in the case that there are edits in each Catalog for any given Photo, the automated system is not at all helpful.  Lightroom would be helpful if it differentiated Photos by the time-stamp of the edits, but it does not.  I _think_ we all agree that it would be best if Lightroom gave the user the option when merging Catalogs with Photos that point to the same file on disk to make the new "master" the Photo that is the most recently changed.  But it doesn't.


----------



## Kirby Krieger (Oct 19, 2018)

JohanElzenga said:


> The newest edit is not necessarily the best one if edits were done in both catalogs.


In general, yes — because in general the edits in situations like this have been made by different people.  In my case, I made all the edits, and I would be satisfied to be able to move forward with the presumption that the newest edit is the one I want to preserve (and have as the "master").


----------



## Kirby Krieger (Oct 19, 2018)

johnbeardy said:


> I've never seen that dialog box, but it's conceivable that it's more recent than my testing.


In the "Import from ... " dialog box, checking "Show Preview" will fly out an extension of the dialog box with a grid displaying the Photos in the container selected in the left sidebar.  If there are changes to a Photo, it will be selected and will show a white border around the thumbnail image.  When you hover your mouse cursor over one of these images, a "tooltip" will pop-up stating "The settings from the imported file will replace the settings of this existing photo."  In the border around each thumbnail image Lightroom shows three additional controls.  The Photo name shows beneath.  A toggle-able checkbox shows on the upper left.  On the upper right there a context-sensitive menu icon (a "hamburger") with an alert symbol (an exclamation point) next to it.  Click this icon and the dialog box I pictured above will slide down just below the bottom of the title bar of the "Import from ... " dialog box.


----------



## Kirby Krieger (Oct 20, 2018)

This post will attempt to explain what happens when you merge two Catalogs with Photos based on the same image file and each Catalog is set to write changes to disk ("Lightroom ▹ Catalog Settings ▹ Metadata ▹ Editing ▹ Automatically write changes to XMP").

There is nothing tricky here — it’s just a bit tangled.

I will present four examples Photos.  In all cases the 2nd Catalog is created from the 1st Catalog and then merged into it with the merge set to make Virtual Copies of Photos when the  groups of settings conflict.  (As shown earlier in this thread, Lightroom always makes the Photo in the source Catalog the master and, when virtual copies are set to be created, makes the Photo in the target Catalog the Virtual Copy.
 - “A” Photo is changed in 1st Cat and not in 2nd Cat
- “B” Photo is changed in 2nd Cat and not in 1st Cat
- “C” Photo is changed in 1st Cat and later in 2nd Cat
- “D” Photo is changed in 2nd Cat and later in 1st Cat

Each Photo in this scenario has three groups of settings:  the 1st Cat settings, the 2nd Cat settings, and the settings on disk (written to the image file that was imported into the 1st Catalog).  Since each Catalog is set to write changes back to disk, two of these three settings are always identical (barring a computer failure).

When the 2nd Cat is created by export from the 1st Cat (or by copying it), all three groups of settings are identical.

Case A:  A becomes A’ in 1st Cat and on disc, on import A is set to master and A’ is set to Virtual Copy; master shows conflict in metadata.

Case B:  B becomes B’ in 2nd Cat and on disc; on import B’ is set to master and B is set to VC; VC shows conflict in metadata.

Case C:  C becomes C’ in 1st Cat and on disc; later C becomes C’’ in 2nd Cat on C’’ over-writes C’ on disc; on import C’’ is set to master and C’ is set to VC; VC shows conflict in metadata.

Case D:  D becomes D’ in 2nd Cat and on disc; later D becomes D’’ in 1st Cat and D” over-writes D’ on disk; on import D’ is set to master and D’’ is set to VC; master shows conflict in metadata.

The reason this is important to me is that I made a mess and now have 28,000 Photos with potential conflicts in their metadata and I don’t know which metadata I want to keep (I want to keep the most recent, but there is no way to tell).  There is a general interest as well:  a good way to leave yourself in a position to resolve conflicts in metadata is to set the Lightroom Catalog preference to always write changes to XMP.  I had done this for years, after I lost important data in a bad crash.  I stopped doing this recently as I searched for performance improvements.  I am now thinking that the “XMP as back-up of Photo settings in Catalog” might be a worthwhile stratagem.  Thoughts?


----------



## johnbeardy (Oct 20, 2018)

If you save xmp from both catalogues, you'll be saving the newest edits each time.


----------



## Kirby Krieger (Oct 20, 2018)

johnbeardy said:


> If you save xmp from both catalogues, you'll be saving the newest edits each time.



Do you advise doing so?  (Also: what does Lightroom do when the file originally imported is a JPG?)

My experience is very limited, but from what I've learned I'd recommend that everyone keep every Catalog set to "Automatically write changes to XMP".  This serves as a potentially work-saving "back-up" of all the changes made to Photos, at the expense of some background processing and disk-writing.


----------



## Victoria Bampton (Oct 20, 2018)

Kirby Krieger said:


> There is one line — it's at the top of p. 503 under the sub-heading "Cons" — where this is mis-stated.  She writes, "_You must keep track of who is working on each chunk of photos, as the latest edits win when merging catalogs._"  In fact, the latest edit does not win.



Oh well spotted! Yes, you're right, the source catalog wins. Thanks, I'll fix that.


----------



## johnbeardy (Oct 20, 2018)

Kirby Krieger said:


> Do you advise doing so?  (Also: what does Lightroom do when the file originally imported is a JPG?)


Absolutely not. I wasn't clear, but was hoping you would see the implications. Imagine you make your best edits in catalogue A, then do something on the same image in catalogue B. What's in the xmp that gets backed up at the end of the day? It's the newest edit, what was only done in B.


----------



## Kirby Krieger (Oct 20, 2018)

johnbeardy said:


> Absolutely not. I wasn't clear, but was hoping you would see the implications. Imagine you make your best edits in catalogue A, then do something on the same image in catalogue B. What's in the xmp that gets backed up at the end of the day? It's the newest edit, what was only done in B.


I'm sorry — I'm probably being a dunderhead, and being dunderheaded I can't tell.  The problem I have in resolving my mistake of editing Photos in two copies of a Catalog is that I can't tell which copy has the most recent edits.  (Lightroom _should_ make this distinction, but it does not.)  

The settings for every Photo are saved in each Catalog that includes the file that was used to create the Photo in Lightroom.  None or one of those settings can be saved alongside (though I don't know how XMP is handled with JPGs, etc.) the file that was used to create the Photo in Lightroom.

If you have two Catalogs with a Photo in each created by importing the same image file, then (afaik) there are three sets of settings saved:  one in Cat One, one in Cat Two, and the settings alongside the imported image file on disk (this set of settings could be null).

I propose, going forward, using Lightroom's "write changes to XMP" specifically because then the most recent changes — from any Catalog — will be on disk.  It's the only way I know of to keep a record of which settings are most recent.  Writing them to disk does not — afaik — cause the settings _in another Catalog_ to be over-written.  Each  Catalog will show those Photos which are not in agreement with the information on disk.

IOW, I think the implications of "write changes to XMP" are beneficial.  But again, I seem to be missing something.

I am assuming that from the "file + XMP" on disk I can faithfully produce both the originally imported file and an image file showing the changes to metadata and development settings stored in the XMP.

In my case, since I want to prioritize the most recent edits, I could simply tell Lightroom to update from disk all the Photos for which the conflict in the metadata is "Changed on Disk".

In general one wants (mho) to prioritize the most recent edits.  I want a workflow, including recovery from mistakes, in which saving and prioritizing the most recent edits is the default.  If I need to go back and over-write the current edits with older edits, that's on me to do semi-manually.


----------

