# Splitting a Lightroom Catalog



## macdane (Dec 8, 2010)

I have 174k photos in my LR3 catalog and am finally getting to the point where I'm seeing serious slowdowns. For the past month or so I've been backing up and optimizing my catalog VERY regularly -- which helps at least temporarily -- but it's time for a better solution. I'd really appreciate having someone familiar with catalog operations take a look at my plan and offer commentary. I shoot a lot of road races (5k, half marathons, etc.) and I'd like to be able to archive old race photos to a separate catalog and then periodically move additional batches of photos there. Here's what I'm planning/wondering about...

1. Backup and optimize main catalog.

2. Identify (by keyword) and select photos from all races more than, say one year old. Here I'd make sure to have all filters other than keywords turned off, so as to be able to grab ALL relevant photos with certainty.

3. From the File menu, choose "Export as Catalog..." and specify a name ("OldRaces") and location. What do I do with the three checkboxes in the export dialog? I want my photos to stay where they are, on whatever drive they're on. All I want to accomplish is to have the information about these photos (including edits, keywords, etc.) stop being kept track of in my main catalog and start being stored in the new catalog. So my gut tells me to leave all three options unchecked. Is that right?

4. I'd expect at this point to have all of my "old race" photos available in both the main catalog and the new (OldRaces) catalog. Right? So after verifying this, I should be able to select the photos again in the main catalog and delete them. In the delete dialog, I'd then select "Remove" rather than "Delete from Disk." That would allow the main catalog to forget about them while keeping the actual image files where they belong, correct?

5. If my plans and assumptions to this point have been good ones, I should now be where I want to be.

6. A year from now I'll want to move all of the 2010 race photos to the "OldRaces" catalog, so I'll repeat the above steps with one addition: after exporting as catalog, I'll switch to the OldRaces catalog and run an "Import from catalog" operation on the recent export of 2010 photos.

Am I on the right track, or do I need to rethink something?

Thanks!
Dane


----------



## Graeme Brown (Dec 8, 2010)

Pretty much perfect, only thing I would change is once you've made your selection add a keyword like "Exported" to all the files selected. Once you've done your export, you can compare totals between the two catalogs without having to worry about re-selecting the right images, and then delete them knowing that you've selected the right ones. It's a minor tweak, basically you're fine.


----------



## macdane (Dec 8, 2010)

Thanks Graeme, that's a great idea.

As a follow-up, since I'm dealing with only catalog info and not actual photos, there's no reason to mount my external drives (where many of the images are located) to do this...is there?


----------



## Kiwigeoff (Dec 8, 2010)

I do separate catalogs for each year and have a master with everything and small previews for a reference.


----------



## MarkNicholas (Dec 9, 2010)

I have a master catalogue with everything in it and a working catalogue in which I do my recent edits and key wording etc.


----------



## macdane (Dec 9, 2010)

Hmm. I'm about 25000 photos into this operation and everything seemed to be going as planned until I just noticed a problem with collections. I have collections and collection sets. The collection sets seem to be exporting/importing fine, but the collections contained within them aren't making it.

Here's what part of 2008 looks like in my main catalog: http://grab.by/7ObA
And here's the same thing in the new catalog: http://grab.by/7Oby

This is a major problem, obviously. Any thoughts would be greatly appreciated. I'm following the steps I outlined above with one exception: when exporting as catalog from the main catalog, I'm checking the first box ("Export selected photos only"). Otherwise, it wants to export all 174000 photos.

Thanks,
Dane


----------



## Victoria Bampton (Dec 9, 2010)

Is it possible that none of the exported photos are in those missing collections?  That could explain why they haven't been recreated.


----------



## macdane (Dec 9, 2010)

Hi Victoria,

No, as of right now, everything shown in the "main" screencap <http://grab.by/7ObA> has gone through the export/import process. That's how the collection sets seen in the "OldRaces" screencap <http://grab.by/7Oby> were generated. If everything had gone as expected, the two screencaps should be identical.

Dane


----------



## Victoria Bampton (Dec 9, 2010)

Just try something for me.  Temporarily change the name of one of those collections, so that the first few digits DON'T match the name of the set and just try exporting those.  I'm wondering if you've found a weird bug.


----------



## macdane (Dec 9, 2010)

I see what you're saying, but no, it didn't work.

Using the 08DD set as my test bed, I removed "08DD-" from the name of each of the 3 collections, exported, then imported to the OldRaces catalog. Same result as before, except now the 08DD set was on its own outside of the 08 set.

So I took your thinking and applied it to the next level, renaming "08" to "2008." Sorry to say this didn't change anything. The "2008" set is nowhere to be found; the "08DD" set remains at the top level of my collections, alongside "08" and "09" and it contains no photos.

Does this give you any more ideas?


----------



## Victoria Bampton (Dec 9, 2010)

Doh.  Ok, an alternative thing to test.  What happens if you right-click on a collection set and export that as a catalog?  Do the collections carry over then?

And are you updated to 3.3?


----------



## macdane (Dec 9, 2010)

The only options I'm given when right-clicking on a collection *set* are:

Create Collection...
Create Smart Collection...
Create Collection Set...
Rename
Delete
Import Smart Collection Settings...

I do see the option to "Export this Collection as a Catalog..." when I right-click on a collection, but I can't do it when selecting multiple collections. I tried it using 2008 > 08DD > First Turn and had the same results as before. The 2008 set doesn't come over; 08DD does, at the top-level of collections, but contains no photos.

I have *not* updated to 3.3 yet. I was thinking I could get this taken care of first and then update once things work as they should. Is there an advantage to updating first?

Thanks,
Dane


----------



## Victoria Bampton (Dec 9, 2010)

Are you on 3.0 or 3.2?  If 3.0, I'd definitely update.  If 3.2, I'd still probably update, as it could be a bug that's since been fixed.


----------



## macdane (Dec 9, 2010)

Argh. I was on 3.2 and just updated to 3.3. Tried again with the exact same results. There has to be a way to do this!


----------



## macdane (Dec 10, 2010)

Update...

After updating to 3.3 and renaming all of my collections and collection sets so there are no instances of parents/children sharing the first few characters of their names, I started over. In the Main catalog, I selected all 25000 photos I've tagged with the keyword "archived" and exported them as a catalog named OldRaces. Perfect. OldRaces has two top-level collection sets: 2008 and 2009. Each of those sets contains several appropriate sets corresponding to specific races. Each race set contains the appropriate collections, and those collections contain the photos they should. Done, right? Nope.

I next switched back to the Main catalog to continue where I left off with the 2009 races. I selected another 6000-ish photos from the next couple of races and exported them to a new catalog called "Temp." I then switched to the OldRaces catalog and imported as catalog from Temp. I now have around 31000 photos in OldRaces, but none of them remember that they belong in collections.

Those 6000 photos represented three races, each in its own collection set under the umbrella of the 2009 set. After importing them to OldRaces, those three sets are at the top level of collections rather than falling under 2009. Not a big deal as I can move them easily enough, but the real problem is that those three sets are empty. Each set should contain several collections which hold the photos from that race, but they contain nothing at all.

Next step, I relaunched LR with the Temp set I had used as an export/import middle ground. (As an aside, why in the world do I even need to do this? Why doesn't LR's Export as Catalog command have the capability to designate a target catalog so I can export directly from Main to Old Races? It seems unnecessarily complicated to have to export to a temporary catalog only to turn around and re-import into the desired catalog. Another question for another day, I guess.) In the Temp catalog, the 2009 collection set is conspicuously absent but the three race-specific sets are there. More importantly, each contains the collections -- and, in turn, the photos -- that they should. So it looks as though exporting from the Main catalog is relatively glitch-free. The missing 2009 set bothers me but doesn't cause any real problem that I can see. The problem seems to be importing from the Temp catalog to the OldRaces catalog. Given that the 2009 set didn't survive the export to Temp, I can't expect it to miraculously reappear in OldRaces. So the import to OldRaces gives me the sets it should but not the contents (collections filled with photos) of those sets.

Does this shed any more light on the issue for the gurus out there?

Thanks!
Dane


----------



## johnbeardy (Dec 10, 2010)

I hadn't commented before but it just confirms my thinking that you should stick with a single catalogue. How much time have you spent on breaking up control of your picture collection? If it's not a problem like you're encountering, there'll be something else. I mean this seriously - it wouldn't be how I would have advised you to go. 

if you are determined to follow this route. why not start a new catalogue and periodically import it to the main one?

John


----------



## macdane (Dec 10, 2010)

Hi John,

First of all, I don't view it as "breaking up control" of my pictures because there's virtually no relationship between the photos I plan to keep in my main catalog and those I plan to break off into a separate catalog. Ultimately, though, I'll get to the point where my main catalog is too large even without all the race photos...what then? Dunno.

Having said that, I've spent very little time breaking my catalog into two catalogs. Where I've spent a TON of time is in trying to figure out why it doesn't work as it should.

Dane


----------



## macdane (Dec 10, 2010)

Ok, I finally caved and called Adobe support. While they were friendly and all, I'm not remotely pleased with the outcome.

Bottom line is that my main catalog appears to be corrupt in some nameless, generic way that regular optimizing and integrity checking wasn't able to catch. When I "export as catalog" photos that are in nested collection sets, I lose the top-level set. Fine. The workaround for that is reasonable. But when those photos are "imported from catalog" into another catalog, virtually all collection info is lost. Not fine.

Over the course of more than an hour on the phone, I created a test catalog, added photos, and put them into collections and sets nested the same way my "real" catalog is set up. From that catalog, I was able to do precisely what I need to do and it worked as expected. I just can't do it with my "real" catalog.

I'm left with no idea what to do. The solution Adobe suggested is to create a new catalog and move all of my photos to it, then recreate the collections. Riiiight. I can see that taking less than a month as long as I don't have any other work to do in the meantime. Completely unrealistic. Alternatively, I can keep going with one big catalog, but that's already a performance problem and only going to get worse.

So, for now, I'm stumped, frustrated and terribly disappointed. Why have untold hours of optimizing, backing up and integrity-checking my catalog utterly failed to catch this apparent corruption? Ugh.


----------



## Victoria Bampton (Dec 10, 2010)

Ok, how about an alternative...  how many collections are there?  How about adding a keyword and using a Smart Collection to group them into a collection.  Less that can potentially go wrong with that.

Other than that, Dan Tull might be up for a look.


----------



## macdane (Dec 10, 2010)

There are something like 400 collections in 80 or so sets. Gotta be honest, though: I don't have a lot of confidence in LR at the moment. I'm reluctant to invest a lot of time rebuilding something that could once again just go "poof" with no warning.


----------



## Victoria Bampton (Dec 10, 2010)

Ouch!  I completely understand your reluctance.  The great thing about using keywords is that they can also be written back to the metadata of the files, so you're not reliant on LR - any DAM should read them.


----------

