# Splitting one catalog into three



## JMay (Apr 19, 2010)

Since my archive of 1'',''' RAW images is only going to get bigger, I feel I should also separate my catalog into several. One will be work photos, another will be personal photos, etc.

How do I bring files and folders with all of the LR adjustments from a current catalog to a new one?

Thank you so much,
Jen


----------



## johnbeardy (Apr 19, 2010)

Don't - and don't worry. Breaking up control of your pictures into multiple catalogues is only likely to cause problems (eg finding pictures, spelling of keywords). And it won't achieve anything. A client has 250k+ images in a catalogue and it's running fine. Optimise your catalogue regularly.

John


----------



## JMay (Apr 19, 2010)

Interesting. LR has been running very slowly for many months now and a computer technician suggested I separate into several catalogs. I've heard that idea from others, too. 

I did just optimize and that seems to help, but I also have a lot of projects coming up this year and will estimate I will be adding 4''' and sometimes 5''' pictures a month. Can adding to a catalog go on indefinitely?


----------



## johnbeardy (Apr 19, 2010)

Yes. my client adds images at about that rate.

To be honest, making separate databases is a lazy suggestion - the kind of solution that seems reasonable for people with little database experience to suggest. So you find some folk with 10000 images imagining they have a big catalogue and must break it up. Others set the bar higher or lower - because they just acting nothing more than a suspicion that it must be better to have smaller catalogues. Quite often slowness is because of something else - bad drivers or 3d settings on graphics cards. While there are valid reasons in some cases for having separate catalogues (eg pictures you wouldn't want the kids to see), 10k images isn't one of them. Optimise regularly.

John


----------



## dj_paige (Apr 19, 2010)

I was just going to say something similar to John.

Let's be clear ... you may have valid reasons for splitting your catalog; reasons that have to do with image content. Only you can make that call.

You do not (in my opinion) have a good technological reason to split your catalog.

Lightroom advice from computer technicians with no Lightroom experience isn't worth much, in my opinion.


----------



## areohbee (Apr 19, 2010)

I agree with John and Paige here. And to add my .'2 - the size of the catalog comes into play during _some _Lightroom operations, and not others. More specifically, the size of the catalog _only_ comes into play when Lightroom is searching for stuff in the catalog, but _not_ if its already found it (Lightroom does not store the catalog in RAM, just the stuff having to do with the photos being presently displayed or worked on). For example, filtering may take a little longer with huge catalogs, but the performance in loupe view and develop mode has _nothing_ to do with the size of the catalog. So, before you go breaking up those catalogs, check out the performance in a 1-picture catalog to see which factors are due to your catalog and which are due to other things. Hope this isn't too much answer for your question.

Summary: In general, the fewer catalogs the better, and if you're thinking about splitting them up, test how much will be gained (or not) beforehand.

PS - There are some users that have developed a workflow that includes a multitude of catalogs, but as John mentioned - there are some significant drawbacks and complications in doing so.

Rob


----------



## JMay (Apr 19, 2010)

Great. Thanks so much everyone for the insight. I am happy not to split the catalog. It was only in hopes of gaining more speed, but optimizing seems to have helped.

Jen


----------



## clee01l (Apr 19, 2010)

[quote author=dj_paige link=topic=961'.msg6492'#msg6492' date=12717'69'5]
I was just going to say something similar to John.

Let's be clear ... you may have valid reasons for splitting your catalog; reasons that have to do with image content. Only you can make that call.

You do not (in my opinion) have a good technological reason to split your catalog.

Lightroom advice from computer technicians with no Lightroom experience isn't worth much, in my opinion.
[/quote] I agree with Paige and John. There is no technical reason to have multiple catalogs. However, there may be a business reason to have two catalogs. 
Let's for a minute assume that you need to separate your personal images from your work images. This might be the only valid reason to have multiple catalogs. 

You ask the question "How do I bring files and folders with all of the LR adjustments from a current catalog to a new one?"

The answer is you export the catalog(s) (and the negatives (original images) if you intend to keep these catalogs separate and on separate machines). At the time of Catalog Export, you have two catalogs that have the same set of Keywords, presets, Collections Etc. But the images in each catalog are different. Now you have two catalogs and your keywords, presets etc are in sync. This is the LAST time they will be in sync unless you spend a tremendous amount of your time manually syncing.


----------



## reinoud (Jul 18, 2010)

[quote author=clee'1l link=topic=961'.msg64933#msg64933 date=1271711282]
There is no technical reason to have multiple catalogs.
[/quote]

I can think of one: backup size. Mu catalog is backed up daily over my ADSL line. If I change just one keyword, the 1.25 GB file is changed and has to get uploaded over my line.
Is there any possibility to split the databasefile into several segments? When only one 1'' MB file would change, my backups would be a lot shorter...


----------



## areohbee (Jul 18, 2010)

You can split your catalog into more catalogs, but you can't split your catalog file into more files.


----------



## Sgt_Strider (Jul 18, 2010)

I thought that LR beings to suffer performance degradation after 1'',''' images? Are you guys saying that on a modern computer (Core i5 75' processor, at least 4GB of RAM, SSD and HDD setup - currently my computer setup), then it's not an issue anymore? What's the limit to the number of photos that LR3 can store in its database? 

Theoretically, can LR3 store at least 1,''',''' photos in its database without suffering from performance degradation on a computer with the kind of spec that I listed above?


----------



## dj_paige (Jul 19, 2010)

[quote author=Sgt_Strider link=topic=961'.msg7'598#msg7'598 date=1279493967]
I thought that LR beings to suffer performance degradation after 1'',''' images? Are you guys saying that on a modern computer (Core i5 75' processor, at least 4GB of RAM, SSD and HDD setup - currently my computer setup), then it's not an issue anymore? What's the limit to the number of photos that LR3 can store in its database? 

Theoretically, can LR3 store at least 1,''',''' photos in its database without suffering from performance degradation on a computer with the kind of spec that I listed above?
[/quote]

There are no firm numbers. And I don't think anyone has done a study on this issue. It depends on your hardware.

At least one user who posts regularly here in the forums says he has a catalog of over 25'K images on a computer that isn't the newest, and he gets acceptable performance, but you do need to optimize the catalog regularly.

So if anyone tells you 1'',''' is the number, tell that person that (s)he is wrong.


----------



## Sgt_Strider (Jul 19, 2010)

[quote author=dj_paige link=topic=961'.msg7'6'3#msg7'6'3 date=1279497652]
[quote author=Sgt_Strider link=topic=961'.msg7'598#msg7'598 date=1279493967]
I thought that LR beings to suffer performance degradation after 1'',''' images? Are you guys saying that on a modern computer (Core i5 75' processor, at least 4GB of RAM, SSD and HDD setup - currently my computer setup), then it's not an issue anymore? What's the limit to the number of photos that LR3 can store in its database? 

Theoretically, can LR3 store at least 1,''',''' photos in its database without suffering from performance degradation on a computer with the kind of spec that I listed above?
[/quote]

There are no firm numbers. And I don't think anyone has done a study on this issue. It depends on your hardware.

At least one user who posts regularly here in the forums says he has a catalog of over 25'K images on a computer that isn't the newest, and he gets acceptable performance, but you do need to optimize the catalog regularly.

So if anyone tells you 1'',''' is the number, tell that person that (s)he is wrong.
[/quote]

No one told me that specifically, but if you do a search on the forums, then I think you'll notice that 1'',''' number is mentioned a lot. That was for LR2.x though and now we're at LR 3.' and with newer hardware, I'm not sure how that would really applies to today. As you can probably tell from my previous post, my computer is pretty new. What's the most number of images anyone has in their LR3 catalog? I'd love to know if anyone here tried 5'',''' or more images in it.


----------



## sizzlingbadger (Jul 19, 2010)

http://www.lightroomqueen.com/community/index.php?topic=73'9.'


----------



## johnbeardy (Jul 19, 2010)

Is 100,000 truly mentioned a lot? If so, it's merely because people like round numbers. LR3 has improved performance with larger catalogues, but there's still no recommended maximum number.

John


----------



## dj_paige (Jul 19, 2010)

[quote author=Sgt_Strider link=topic=961'.msg7'6'9#msg7'6'9 date=12795'6848]
... but if you do a search on the forums, then I think you'll notice that 1'',''' number is mentioned a lot. 
[/quote]

If you do a search on the forums, you will also see that people say there is no maximum number


----------



## clee01l (Jul 19, 2010)

[quote author=reinoud link=topic=961'.msg7'588#msg7'588 date=1279489'16]
[quote author=clee'1l link=topic=961'.msg64933#msg64933 date=1271711282]
There is no technical reason to have multiple catalogs.
[/quote]

I can think of one: backup size. Mu catalog is backed up daily over my ADSL line. If I change just one keyword, the 1.25 GB file is changed and has to get uploaded over my line.
Is there any possibility to split the databasefile into several segments? When only one 1'' MB file would change, my backups would be a lot shorter...
[/quote]I would still not call that a technical limitation. A hardware/Process consideration but not a technical limitation.Backups should be transparent to the user. I use Carbonite over DSL for one of my automated backups. Some of the files that it backs up are over 1GB. I honestly don't know how long these files take to back up and I don't care. It happens transparently in the background. FWIW, my LR catalog (singular) is not one of the files that Carbonite backs-up. There is a possibility of corruption in backing up an open active file like the lrcat. I choose instead to back up my lrcat backup file. So that I will always have at least one generation backup off-site if it is needed.

I would suggest that instead of complicationg your LR workflow with multiple catalogs, you look instead into simplifying your backup process. If your backup process is not transparent, then you should investigate ways to make it so.


----------



## Deleted member 8190 (Jan 17, 2011)

Despite recommendations to the contrary , I just split up my 7000ish  photo catalog into 2 Catalogs based on a boundary that makes sense for  my workflow. I did this because it was taking an excessively long time  to edit photo's as when I clicked on a photo the editing controls would  be grayed out for minutes at a time. After reading and trying many,many  other optimization techniques that help a little ( google "lightroom  performance" or "lightroom optimize" for those techniques ) I broke down  and committed a sin...I split up my catalog.

I tested the effect by exporting a few files as a catalog and then  importing them into a new catalog. The performance improvement was  incredible! 

Note that I am using slow hardware while on travelling assignment, so  the faster system may have a higher threshold than 5k pics, while a  slower system may have a lower one. 

Moral of the story, there isn't a single answer as to whether or not you  should split up your catalog and when you should do it, but I know for a  fact that a large library can be a bottleneck to system performance.


----------



## MarkNicholas (Jan 17, 2011)

I have created a working catalogue in which I do all my edits. In that catalogue I only have the current photos I am working on. This working catalogue has the exact same keywords, pre-sets etc as my "master" catalogue. When I have finished editing etc. I import these photos into my "master" catalogue and then remove them from the working catalogue. If for any reason I want to carryout further edits on old photos I just imprt them into my working catalogue then do the edits and then import them back into my master catalogue.


----------



## johnbeardy (Jan 17, 2011)

jagooch said:


> Moral of the story, there isn't a single answer as to whether or not you  should split up your catalog and when you should do it, but I know for a  fact that a large library can be a bottleneck to system performance.


What will actually be responsible for the improvement will be the optimisation/integrity checks (which happen with new catalogues or by running the File > Optimise command), conceivably any driver updates you may have made (bad drivers being the usual cause of slowness), and you yourself point to the bottleneck being slow hardware. 7k or 5k are trivial numbers of photos for Lightroom, even on an old computer, and people reading this thread should not be misled - splitting catalogues simply fragments your control over your pictures and your ability to work efficiently with them.

John


----------



## Bruce J (Jan 17, 2011)

MarkNicholas said:


> I have created a working catalogue in which I do all my edits. In that catalogue I only have the current photos I am working on. This working catalogue has the exact same keywords, pre-sets etc as my "master" catalogue. When I have finished editing etc. I import these photos into my "master" catalogue and then remove them from the working catalogue. If for any reason I want to carryout further edits on old photos I just imprt them into my working catalogue then do the edits and then import them back into my master catalogue.


 
Mark, I've been interested in doing something like this for awhile, but can't figure out a good mechanism for keeping the keyword lists synced between two catalogs.  How do you manage that part of the process?  Thanks,


----------



## dj_paige (Jan 17, 2011)

Bruce J said:


> Mark, I've been interested in doing something like this for awhile, but can't figure out a good mechanism for keeping the keyword lists synced between two catalogs.  How do you manage that part of the process?  Thanks,



Mark, and Bruce J ... allow me to translate this workflow that Bruce is admiring and Mark says he is actually carrying out into "PaigeSpeak".

Here is my translation.

This workflow uses an awful lot of manual work that you don't have to do, that Lightroom makes easy for you to do if you simply stick with keeping your photos in their original catalog (or if you use Catalog operations instead of import and export of photos). I fail to see the benefit of constantly importing and then exporting photos, re-importing photos, maintaining keywords lists that are matched in multiple catalogs, and so on.


----------



## Bruce J (Jan 17, 2011)

dj_paige said:


> Mark, and Bruce J ... allow me to translate this workflow that Bruce is admiring and Mark says he is actually carrying out into "PaigeSpeak".
> 
> Here is my translation.
> 
> This workflow uses an awful lot of manual work that you don't have to do, that Lightroom makes easy for you to do if you simply stick with keeping your photos in their original catalog (or if you use Catalog operations instead of import and export of photos). I fail to see the benefit of constantly importing and then exporting photos, re-importing photos, maintaining keywords lists that are matched in multiple catalogs, and so on.



Yes, I have seen this opinion voiced here many times by you and by others.  In general, I agree, and I do have a single catalog w/ ~50k images.  I realize that others have bigger catalogs, but I think the factor that most people who don't understand why one would want to work in a smaller catalog don't take into account is that some people don't work on the latest equipment.  If I had a new, fast, 64-bit machine, I would never consider the added work of having a working catalog as well as an archive catalog.  But, if it speeds up my everyday processing by only a few percent, it's certainly worth the added overhead of maintaining two catalogs and moving images from the working catalog to the archive catalog, as needed.  I suspect that, if you worked every day on my hardware and OS, you would be looking for ways to speed things up also.  Easy enough to say that one should get faster equipment, but I haven't noticed anyone around offering to donate same.

In experimenting w/ an approach like this in the past, the sticking point for me has always been keeping the keyword list the same in each catalog, so I was inquiring of Mark whether he had a found a simpler way of achieving that than I have.  I certainly don't find myself admiring that workflow or advocating it for anyone else; just interested in learning.  Sorry if you find that offensive in some way.  Cheers,


----------



## dj_paige (Jan 17, 2011)

There are workflows that don't involve importing and exporting photos, and that don't involve a human being maintaining keywords lists in separate catalogs, that allow people to work with smaller catalogs on older equipment. The approach involves creating a small catalog using catalog operations (not photo import and export) to use on a different computer. Specifically, I am referring to File->Export as Catalog to create a small catalog that can be transferred to another computer, and then File->Import from Catalog to merge everything together back on the main computer. This _could _speed up your workflow, and doesn't involve the manual intervention that photo import and photo export workflows would involve.

My philosophy is let the computer hardware and software do as many tasks as possible. I abhor workflows that involve large amounts of human intervention, when alternative workflows that involve much less human intervention are available.


----------



## johnbeardy (Jan 17, 2011)

Bruce - until recently I worked on an older slower machine - the "don't fragment control or your workflow" advice was the same. You can't sync keywording lists, without doing it manually, and that's just one of the inefficiencies that's needlessly introduced! There is something to be said for having a temporary "inbox" catalogue for new work that you're still processing, but not much, and next to nothing to be said in favour of splitting a master catalogue.

John


----------



## JMay (Apr 19, 2010)

Since my archive of 1'',''' RAW images is only going to get bigger, I feel I should also separate my catalog into several. One will be work photos, another will be personal photos, etc.

How do I bring files and folders with all of the LR adjustments from a current catalog to a new one?

Thank you so much,
Jen


----------



## Bruce J (Jan 17, 2011)

dj_paige said:


> There are workflows that don't involve importing and exporting photos, and that don't involve a human being maintaining keywords lists in separate catalogs, that allow people to work with smaller catalogs on older equipment. The approach involves creating a small catalog using catalog operations (not photo import and export) to use on a different computer. Specifically, I am referring to File->Export as Catalog to create a small catalog that can be transferred to another computer, and then File->Import from Catalog to merge everything together back on the main computer. This _could _speed up your workflow, and doesn't involve the manual intervention that photo import and photo export workflows would involve.
> 
> My philosophy is let the computer hardware and software do as many tasks as possible. I abhor workflows that involve large amounts of human intervention, when alternative workflows that involve much less human intervention are available.



Sorry if I have caused some confusion.  I couldn't agree more.  The workflow I was referring to, and that I find interesting is to use 'Export as Catalog' and 'Import as Catalog' to move between a small, current work catalog and a much larger archive catalog.  I agree about the advantages of maintaining all of ones images in a master catalog.  That's where I do all of my searching, printing, exporting, etc.  However, I have found my copy of LR on my hardware to be more responsive in the Develop module when working w/ a very small catalog (100's of images).  So, in the past, I have toyed w/ a workflow that involves importing new images into a working catalog, processing them there, and then exporting those new images from the working catalog as a catalog and importing that small catalog into my master catalog.  The problem I have run across is that if I introduce a new keyword into the working catalog, it is not copied to the master catalog when I go through the 'Export as Catalog', 'Import as Catalog' process.  From Mark's comments, I thought it was possible that he had found a way around that bit of the process and was curious as to what he was doing.  I haven't used this process in several versions of LR, so perhaps it has changed since the last time I tried it.  I'm certainly not advocating this workflow for anyone else, and possibly not even for myself, just curious how others are getting around that particular problem.


----------



## dj_paige (Jan 17, 2011)

Bruce J said:


> The workflow I was referring to, and that I find interesting is to use 'Export as Catalog' and 'Import as Catalog'



Oh, well that's different! 



Bruce J said:


> However, I have found my copy of LR on my hardware to be more responsive  in the Develop module when working w/ a very small catalog (100's of  images).



Hmmm ... the number of images in the catalog should have virtually zero effect on the speed of the Develop module.



Bruce J said:


> From Mark's comments, I thought it was possible that he had found a way  around that bit of the process and was curious as to what he was doing.



Yes, his "workaround" is to actually do more work, and to use the features of Lightroom less.



Bruce J said:


> The problem I have run across is that if I introduce a new keyword into  the working catalog, it is not copied to the master catalog when I go  through the 'Export as Catalog', 'Import as Catalog' process.



I haven't tried introducing a new keyword in a working catalog, and then moving that working catalog into a master catalog via Import From Catalog. I am very much surprised that you are seeing this, as it seems like a stupid thing to be programmed into Lightroom, it kind of defeats the purpose of Import from Catalog; and I wonder if anyone has seen this. Or perhaps you could try it again in a recent version of LR.

If I could make a suggestion, Mr. Bruce J, if you tell us your needs, we can probably answer your questions better than if your needs dribble out over several posts.


----------



## RikkFlohr (Jan 17, 2011)

> I haven't tried introducing a new keyword in a working catalog, and then moving that working catalog into a master catalog via Import From Catalog. I am very much surprised that you are seeing this, as it seems like a stupid thing to be programmed into Lightroom, it kind of defeats the purpose of Import from Catalog; and I wonder if anyone has seen this. Or perhaps you could try it again in a recent version of LR.


 
The only time I have seen this happen is when a keyword isn't assigned to an image. The only keywords that migrate during an Export as/Import As catalog operations are those assigned to a photo. If you are seeing assigned keywords not migrating, I would like to investigate more.


----------



## MarkNicholas (Jan 18, 2011)

Bruce J said:


> Mark, I've been interested in doing something like this for awhile, but can't figure out a good mechanism for keeping the keyword lists synced between two catalogs. How do you manage that part of the process? Thanks,



Bruce, when I created my working catalogue I imported my keyword list from my master catalogue such that it was initially identical. As and when I add new keywords it does require a little manual management to keep the keywords in my working catalogue and master catalogue synced but this literally takes just a few minutes. I use a hierarchical system and just drag new keywords into the appropriate parent.

With my relatively ancient computer I notice a huge improvement in performance when in my working catalogue. For others with more power computers they may not notice so much.

I have been using LR since LR1 have processed over 20,000 photos in LR and for me using a working catalogue is by far the best work flow that I have adopted. The time saved in working in an empty catalogue far exceeds the time taken to keep my keyword lists in sync and importing from the working catalogue to the master.

There is no harm in giving it a try. After all its just a duplication of whats already in your master catalogue.


----------



## Bruce J (Jan 18, 2011)

Thanks Mark, that is the information I was looking for.  No magic bullets; you are essentially doing what I have done in the past.  Cheers,


----------



## Bruce J (Jan 18, 2011)

RikkFlohr said:


> The only time I have seen this happen is when a keyword isn't assigned to an image. The only keywords that migrate during an Export as/Import As catalog operations are those assigned to a photo. If you are seeing assigned keywords not migrating, I would like to investigate more.


 
Rikk - As I mentioned, I haven't done this in a few LR versions.  I'll give it a try again when I get a chance.  Thanks,


----------



## teeroy97 (Jan 19, 2011)

I had the same issue and this was very informative.  Thanks everyone.


----------

