# Lost edits



## jerry12953 (Apr 30, 2010)

I'm sure my problem is closely related to many others on here, but I can't relate the replies exactly to it.

I've lost all the edits to the images on my external hard drive. I can open them, but the history pane just says "Import '8/'2/1'".

That may well have been the date I moved everything over to a new PC, but I can't quite remember.....

To get access to the edits I can go to my lrcat backups and use one dating before '8/'2/1', but I can't, for example, add them to a collection started since them. And it's a pain, of course.

At one time the external HD had two letters "I" and "J" in the folders panel, but "I" has disappeared. 

How can I solve this one?


----------



## dj_paige (Apr 30, 2010)

You re-imported your photos into Lightroom. Re-importing is almost never a good idea. 

This is the cause of losing your edit histories, which are stored ONLY in the Lightroom catalog, along with other information that is ONLY stored in the catalog. Re-importing means that Lightroom thinks you have a brand new photo that it has never seen before, and thus has no edits.

If you restore one of the backup catalogs, you should be able to view your edits and reconnect the photos, if necessary. For collections started after this date, you might have to merge (Import as Catalog) your newer catalog into the restored backup. I'm not sure that will work as I have never done this, maybe someone else can speak up.

In the future, no re-importing. The proper way to handle moves to a new PC is to move your catalog file(s) and photos, keeping the same folder structure of your photos, and then just double-click on the catalog file and if necessary, reconnect the photos/folders.


----------



## jerry12953 (Apr 30, 2010)

Hi,

Thanks for replying.

So what you're suggesting is this -

I go back to a pre-'8/'2/1' catalogue, click on that, which opens LR -yes?

Then I have to bring everything more recent .....new images and all the edits done to them......into that ?

You think I might be able to do that by using "Import as catalogue"?

I'm just trying to get everything straight in my head before I do anything too drastic......LR seems to have a peculiar kind of logic which I find it difficult to comprehend.....


----------



## Jim Wilde (Apr 30, 2010)

I think it might be easier to do it the other way round....i.e. try to capture all the metadata from the pre-'8/'2/1' catalog and bring them into your current 'master' catalog.

Paige, I think this is actually the exact same process as we were discussing in Newmarket2's Elements thread earlier today. The process would "simply" be to open the pre-'8/'2/1' catalog, select all photographs, then export the metadata only to a 'temporary' catalog. Then relaunch Lightroom with the current master catalog and run an "Import from Catalog" process, specifying the temporary catalog as the input. That should cause the metadata of all 'matching' images to be updated with the "old" metadata. 

The issue, before starting any of that, would be to ensure that the folder structure from BOTH master AND pre-'8/'2/1' catalogs is 1''% aligned....if not I'm not sure what might happen (this means careful backups of BOTH before starting!).

Before we do anything, can you post a screenshot of the top-level folder hierarchy as seen in the Folders panel of BOTH catalogs? If we can get them lined up, this might be an easy fix.

What do you think, Paige? Denis?


----------



## Denis Pagé (Apr 30, 2010)

Seem to be a good plan. With backups for sure!


----------



## jerry12953 (Apr 30, 2010)

OK! But first of all how do I post a screen-shot?


----------



## jerry12953 (Apr 30, 2010)

just two other things......

A) There's a lot more folders and files in the J drive now than there was on '8/'2/1'.

B) I've been reading round a bit and I just wonder if the problem could have been caused by the J drive being renamed the I drive at some stage? I can remember being puzzled by the fact that the ext HD was called both I and J. I read that windows can do that for you.


----------



## Jim Wilde (Apr 30, 2010)

Hello Jerry,

Picking up on some of your points:

1. To attach screenshots, click on the + at the side of Additional Options when previewing your post and you will see the option to browse to and attach a small (256kb max) picture.

2. "There's a lot more folders and files in the J drive now than there was on '8/'2/1'."...does this simply mean you've added a lot of pictures/folders since then (not a problem) OR do you mean that there are more pre-'8/'2/1' folders in the current catalog than were in the backup catalog (which could be a problem).

3. Drive letters....yes they can change, but if we are dealing with only one external drive it probably means that on your old PC the drive letter changed and you either didn't notice or didn't understand the implications. The consequences could well have been that some folders were imported into Lightroom when it was, say, Drive I...and after the change subsequent folders were imported as Drive J. Lightroom will happily show both drives in the Folders Panel, but only the current drive letter would have 'the green light', and all folders logged under the I drive would have had ? against them (meaning Lightroom cannot locate the folder because it's on a drive letter that no longer exists).
When you moved to your new PC, I suspect you created a new catalog (rather than restoring the catalog from the old PC), then imported all your picture folders anew...this would explain why now only one drive is shown in the Folders Panel (which is good), but you've lost all the edits that you did prior to the switchover (which is bad).

Does that make sense?

If you could progress the screenshots that would be great (you may need separate posts) and if you could also include in the same shot a Library Filter breakdown of the total number of pictures in the catalog by year upto 2'1' and by month for 2'1', that would be most useful. See my attached screenshot.

Yell if any of this is not clear.


----------



## jerry12953 (Apr 30, 2010)

Hi Jim,

Its very kind of you to take time out to help me with this.

1) Excuse my ignorance but I don't know how to actually do a screenshot! So it may take me a while to sort it out.
2) There are 5885 photos on the J drive now as opposed to 1571 on 8/2/1'. I'm not sure how they got there, although i have a horrible feeling it may be your second possibility. (ie bad).
3)As far as I can remember I used a file transfer cable to move all the stuff from my old PC to the new one, and re-installed LR on the new one. I may have re-imported the files from the ext HD, which I now realise was "a very bad idea"!

I'll come back with a screenshot asap.

jerry


----------



## Jim Wilde (Apr 30, 2010)

Jerry, 

If you are using Win7 you have a utility called 'Snipping Tool' which you will find in All Programs&gt;Accessories. You can take a screenshot with that.


----------



## jerry12953 (Apr 30, 2010)

Thanks for that, but in the meantime I'd worked out how to do a screenshot using the keyboard control.

I'll attach the first ('8'21') with this post.


----------



## jerry12953 (Apr 30, 2010)

2nd screenshot attached


----------



## Jim Wilde (Apr 30, 2010)

Jerry, thanks for those screenshots, but could I ask you to do them again with a couple of small changes.

In the Library Filter you can make the display area larger by grabbing the bottom bar and dragging it downwards (I'd like to see an exact year by year photo count so that we can see what differences there are in the two catalogs). 

Also in the '8'21' catalog can you collapse the Raw Files top level folder so that I can see what drives are listed further down?

Thanks.

BTW, I see you have some shots at Bala....coincidentally my sister has a caravan there and I was talking to her there today. I have a small Bala folder as well!


----------



## jerry12953 (Apr 30, 2010)

I don't think there's any more date info for 2'1'. Do you need it for earlier years?

Regarding the J drive, it has a very long list of folders which need scrolling through. Not sure how I can do a screenshot of it.....

To be honest its getting a bit late for me to be doing this sort of thing...could we continue tomorrow morning?

Perhaps by phone might be better?


----------



## Jim Wilde (Apr 30, 2010)

No worries Jerry, I'm obviously not making myself very clear.

Will send a more detailed note tomorrow.

Cheers.


----------



## jerry12953 (Apr 30, 2010)

Cheers,

I'll log on again after breakfast!

jerry


----------



## Jim Wilde (May 1, 2010)

Jerry, picking up were we left off last night....

Summarising were we got to:

1. You have lost the edits to all your pictures pre-'8'21'.
2. We think they are in a backup taken before '8'21'.
3. We think that if we can export these edits as a catalog, we can then import into the current master catalog and bring all your edits back.
4. There is confusion over drive lettering somewhere in the mix.
5. Before we can proceed we need to ensure that the pictures in both catalogs are properly "aligned" from a drive letter perspective, otherwise we are wasting our time and potentially making things worse.
6. Hence the need for screenshots of the Folders Panel and the Year by Year picture numbers.
7. So far the screenshots you have sent are raising more questions than providing answers, so I would ask you to repeat them as follows:

A. Start with the '8'21' catalog. Drag the bar at the bottom of the filter panel downwards to expand the panel so that ALL the years are showing (see the example screenshot that I sent you). 
In the Folders Panel (where we can already see that you have some "missing pictures") I would like to be able to see all the drives this catalog thinks it has images on.....I can see you have a total of 3'54 out of 4671 on the C drive, where are the rest. Just click on the down-arrow next to Desktop and Raw Files in the Folders Panel and those lists will collapse, allowing us to see what is further down the list.

B. On the current catalog, repeat the action on the filter panel so that we can see all years. And in the Folders Panel, collapse the list below the J: drive so that we can see where the rest of the 9593 images are.

We already have one glaring issue in that there is a large discrepency between the two catalogs in terms of the overall numbers in each. You may be able to explain that.

Have a go at the new screenshots and then I'll get back to you with some thoughts about where to go next.


----------



## jerry12953 (May 1, 2010)

Hi,

1)yes
2) They definitely are in pre-'8/'2/1' backups. Looking at the size of the backups (in MB) they vary quite considerably. There's a more or less gradual increase in size up to 63.1MB on 8/2/1', then it jumps to 93.6MB....... It rises gradually then until 27/'4/1' when it jumps from 1'6 to 167.

I suspect that it was at this point that I realised there was a problem, and I almost certainly DID reimport the J drive into LR. Since then it is either 167 or 63.7 - presumably depending on whether I'm using the current catalogue or the '8'21' one. But twice it's 93.8 , which is a bit puzzling? I'm fairly sure when I realised that re-importing didn't help, I deleted the re-import, by the way.
3) OK.....
4) yes
5) OK
6) OK
7) OK

I will have another go at the screenshots now, but there wasn't any further information in the Library file for 2'1'. 

jerry


----------



## jerry12953 (May 1, 2010)

Hi,

have done the first snip (screenshot wouldn't work &gt;:() and am attaching it.


----------



## jerry12953 (May 1, 2010)

2nd snip attached,

jerry


----------



## Jim Wilde (May 1, 2010)

[quote author=jerry12953 link=topic=9715.msg65789#msg65789 date=12727'4922]


I will have another go at the screenshots now, but there wasn't any further information in the Library file for 2'1'. 

jerry
[/quote]

Thanks for the screenshots, the Library Filter panel view was just what I wanted to see. However, I'm still not seeing the view of the Folders Panel that I need to see....the most important information first is the high-level view of the disk drives and any associated missing files. I need you to collapse the view of the sub-folders (we can expand them again later if we need to drill down to that level), so that ALL the disk drives (and their respective totals) can be seen in the Panel at the same time. Do you understand what I mean? If so can you retake the screenshots of both catalogs with that information showing.

As you will have noticed, there are significant year by year differences in the totals in the two catalogs....do you have any idea at this stage why that is? And ignoring the last 3 months of this year, as these are obviously not in the older catalog, which of the two catalogs would you say better represents your Lightroom library?


----------



## jerry12953 (May 1, 2010)

You mean just the four basic numbers alongside C, D, E and J?

Try again......

Sorry to be so thick.


----------



## Jim Wilde (May 1, 2010)

Almost there..... 

If you get a minute, could you repeat the two screenshots but this time also include the Library Filter display (having all the data in one picture will save me from having to keep moving between the various shots that you've given me). And in the '8'21' shot can you also open the Catalog panel (just above the Folders Panel) as you did in the current catalog shot.

And have you had a chance to think about why the totals for previous years are so different....do you by any chance "remove" originals from the catalog but NOT choosing the "delete from disk" option, i.e. have you managed to reimport a load of pics that you've previously rejected but not deleted?


----------



## jerry12953 (May 1, 2010)

Try again.....

Am attaching latest snips. I hope these are the right ones.

I'm really not sure why the totals are so different. It's possible that some (or even many) of these images were never in Lightroom, just stored in the J drive with the ones that were. I'll have to think about it some more.

I'm fairly sure that when I delete files I get rid of them completely. But again I just don't know. One never thinks about these things when everything is hunky dory.

There seems to be some major shortcomings in the way Lightroom works. I wonder if they'll fix them in v3?

jerry
jerry


----------



## Jim Wilde (May 1, 2010)

Jerry,
The last screenshot was incomplete as you didn't have All Photographs selected when you displayed the Library Filter, so the year by year count is wrong. Could you do that one again please? Ta.

Re your point about Lightroom's "major shortcomings", I would tend to disagree with that. Yes, it is easy to get into trouble if you don't fully understand how it fundamentally works....but that is true of most database software. The key is to get that understanding first before you lead yourself down a wrong path....and version 3 will not change things as the underlying principles of the "asset management" aspects will remain largely the same.


----------



## jerry12953 (Apr 30, 2010)

I'm sure my problem is closely related to many others on here, but I can't relate the replies exactly to it.

I've lost all the edits to the images on my external hard drive. I can open them, but the history pane just says "Import '8/'2/1'".

That may well have been the date I moved everything over to a new PC, but I can't quite remember.....

To get access to the edits I can go to my lrcat backups and use one dating before '8/'2/1', but I can't, for example, add them to a collection started since them. And it's a pain, of course.

At one time the external HD had two letters "I" and "J" in the folders panel, but "I" has disappeared. 

How can I solve this one?


----------



## Jim Wilde (May 1, 2010)

Jerry, 
Have done a bit of analysis of the two catalogs, and the differences are quite large. Discounting Feb to April this year, your current prime catalog has *4328* more images than the '8'21' catalog. MOST, but not all, of the differences can be seen in years 2''6 (494 more), 2''7 (1427 more) and 2''8 (15'7 more)....and there are also 853 more images with an "unknown" date.

Looking at camera bodies, the major areas of differences are your 2'D (1519 more), your 4'D (1516 more), your scanner (3'1 more), and 974 more images with an "unknown" camera type.

This goes WAY beyond the original premise that you've reimported your images into a new catalog and thereby lost all the edits, and quite what all this means is difficult to say, and probably only YOU can say. The bottom line is that you now have 4'''+ more images in your catalog than you did in February, and you need to decide which, if any, belong in the catalog, or if they are surplus to requirements.

Before we do anything, I think you need to spend some time looking through the current prime catalog. In particular try to establish what all the 'unknowns' are (click on the word in the filter and LR will populate the grid for you with the appropriate images). *What we need to do is establish the integrity of the catalogs to determine which of the 2 you feel more accurately reflects the library that you wanted as at the '8'21' cut-off.* The answer to this question will determine what the best way forward is. Also spend some time looking at the 2'D and 4'D images and deciding what needs to be done about those, as the vast majority of pictures taken with those cameras do not appear in the '8'21' catalog....is there a valid reason for this?

From where I sit, BOTH catalogs have some issues to resolve regarding missing files (though thankfully these are not numerous), but we can deal with those when we decide which catalog we want to use going forward.

Have a good look around and come back with any more questions or observations.


----------



## jerry12953 (May 1, 2010)

I have compared the back-ups for 15.35 hrs '8'21' and 2'.4' hrs '8'21'.

Most of the additional files are TIFFS, there's a smallish number of JPGs but there are also about 16'' RAWs which were added straight in to the RAW file folders. This was out of a total of 4'5' new files.

It was between those two backups that the name of the external HD changed from J to I. Afterwards all the files and folders had a ? on them. Could this be a clue to what happened?

I've also noticed that on 27/'4/2'1' the back-up jumped to about 163 mb. I think I must have re-imported a whole bunch of files just priior to that. You will notice, if I have done the screenshots properly, that both I and J drives are present. I think the clues to what has happened might be on 8/2 and 27/4.

I accept what you're saying about databases, but not everybody understands how databases work. Nor do they want to know!

 l've had another go at those screenshots. But I'm going to have to leave this for a while then. I've go to go out and do some photography! I'll come back to it on Wednesday. If you can shed any light on it while I'm away that would be great!

PS just read your latest post, and I will go into it in detail when I get back. Thanks.

cheers, jerry


----------



## Jim Wilde (May 2, 2010)

Jerry,

Have had a look at the latest screenshots, and to be honest it is difficult to be certain about what might have happened to create what now appears to be a bit of a mess (understatement). I've tried to imagine a sequence of events which could explain the differences between the two '8'21' catalogs, the following is one but there are no doubt others. Assuming that there is ONLY one Lacie external drive in the mix, and assuming it was drive J at the start of the day, then....

1. At 1535 you have a catalog with your pictures spread around your C drive and the external J drive. The J drive contains approximately 5''' picture files, but ONLY 1571 are in your catalog.
2. At some point after that, the drive letter for the Lacie changed to I (don't know why, there will be a reason but doubt if we'll find out).
3. Starting Lightroom at this point you would have had ALL the files on the Lacie(J) show up as missing.
4. You could then have seen that the drive letter had changed, so you could have (within Lightroom) removed all the 1571 images from J and then done an import from I, bringing in all 5'''+ images on it, rather than just the 1571 you originally had.
5. Then the drive letter changed back to J, and at that point that's the 2'4' catalog, i.e. the J drive is now active again but has ' pictures (because they've been removed) and the I drive is inactive (because it now doesn't exist) so you have 5'''+ 'missing files'.

As the current catalog does not show the I drive, then I assume at some point you removed it from the catalog, and then did another re-import from the J drive to get to the current situation.

Regarding 27/4, that I drive is NOT the Lacie, but a removeable drive (e.g. a USB stick)....I assume you were maybe trying to rectify the situation by importing some 54'' images from a backup, but subsequently deleted them again as they do not appear in the current catalog.

I don't know how much of the above resonates with your memory, but if any of it does then there have clearly been a few mistakes made along the way. Water under the bridge at this stage, but you really should spend a little time getting to grips with your import workflow. No you don't have to be a database expert, and that's not what I was suggesting, but a solid understanding of some of the Lightroom fundamentals is, IMO, absolutely essential....especially if you are making a living out of your photography. There are plenty of good books out there that would help, and there are LOADS of free resources in these Forums to help. It would be worth spending some time having a look around.

Sorry, here endeth the lecture....  

Going forward, please refer back to my previous post and help us decide how best to help you. If you liked the first '8'21' catalog (with the 1571 images on J), we can go back to Paige's original suggestion of using that as a new 'master catalog' and then extract all the Feb,Mar,Apr pics from the current catalog to merge with the new master.

Take some time looking at the current catalog, as I suggested, then get back to us with your thoughts.

Hope you are enjoying your current assignment.....I think I need to get out more as well.


----------



## jerry12953 (May 6, 2010)

Hi jim,

I'm just getting my head around the lightroom problem again (or trying to), and thought I would just pass on to you this suggestion, which has come from a photography forum.

_"My suggestion would be to open up your backed up lrcat file, this should give you all edited pictures prior to the date of the back up.

If this is the case, then go back into your main catalog and remove all of the images from prior to the back up from your new catalog (remove from catalog, but keep files). Then use the "Import from catalog" feature to import the images and edits from your backed up catalog. _ 

Is this basically what you're suggesting as well? I'm sorry to be a simpleton, but I find it hard to grasp what some of these manouvres actually mean!

I've had another thought about how the problem arose. I transferred all my files to the new PC on January 26th, so the 8th Feb date does not relate to that. I suspect it may have taken place when I took the machine into a local computer shop because i couldn't get my accounts software to work on 64bit windows 7. 

It would be great to think it wasn't my fault, although of course the resultant mess obviously was. I'm pretty sure I did re-import the contents of the J drive, and then delete them again on 27'41'. (or something like that)

If I went back to the '8'21' 15.35 catalogue, and used it as the basis for going forward,

a) I wouldn't lose the RAW files; they would still be on the relevant HD?
b) I could still import them into LR from there?
c) would I be able to delete all the LR backups dating from then until now?

cheers,

jerry


----------



## Jim Wilde (May 6, 2010)

Hi Jerry,

Just a quick response to acknowledge your post and to let you know I'll do a proper response this evening.

Hope you had a good trip. 

Cheers.


----------



## jerry12953 (May 6, 2010)

Thanks.

jerry


----------



## Jim Wilde (May 6, 2010)

Jerry,

That suggestion from the photography forum is one of several alternative methods of creating what will effectively be a merged catalog. Not including all the new pictures that you have taken and imported since '8'21', there are approximately 43'' files (which are on the J drive) which are now in your master catalog that were NOT in the master catalog at 15.35 on '8'21'. Although there will be variants and additional backup steps, I currently see 3 main scenarios:

Alternative 1: *You decide that you DO NOT want ALL of those 43'' “extra” files in your catalog*, in which case we would use the '8'21'-15.35 catalog as the new master, we would export from the current master all the new post-'8'21' pictures into a temporary catalog, which we will then import into the new master.

Alternative 2: *You decide that you DO want all of those 43'' “extra “ files in your master catalog*, in which case we will retain it as the master, then we will export all the metadata from the '8'21'-15.35 catalog for importing into the current master (i.e. getting those “lost edits” back).

Alternative 3: *You decide that you do not want ALL, but do want SOME, of those 43'' “extra” files in the master catalog*. Definitely a more complicated recovery, and will largely depend on how easy it is to identify and isolate the particular “extras” that you want. More difficult, but by no means impossible.

Which recovery option we use will depend entirely on your analysis of the current master catalog as per my post on May 2nd.

*One other question: we have concentrated mainly on the issues relating to the J drive....however I also note that you have some 3'''+ files on your C drive....so are the edits missing for these files also, or just the 1571 pictures on the J drive?*

The answers to the 3 questions at the end of your post this morning:

a)	We are NOT going to (intentionally) delete ANY of the raw files from either HD unless you specifically request it. And if we are careful and consider our approach thoughtfully we will not unintentionally delete any files either!  
b)	Yes....if they still exist we can still import them if that is what you subsequently decide you want to do.
c)	Yes.....but certainly NOT until we are happy that we have rectified the current situation and taken a series of new backups. After that we can look at your backup strategy (and a few other things). But that’s for later, not now.

Hope that all made sense to you. Now just waiting on you to complete the analysis of your current catalog and decide which of the three alternative scenarios you would like to go with.

Cheers!


----------



## jerry12953 (May 6, 2010)

Thanks very much for that incredibly detailed response and list of suggestions.

I think your alternative no2 is the one I would go for. I would have been quite happy to continue as I was had I been able to find the edits to the images on the J drive.

No 1 seems a bit of a backward step, and I can't see the point of no3. Might as well have them all.

I'm glad to say that there appears to be no problem with the files or edits on the C drive. That had been my "working area" for importing new images and working on them for the last year or so. Those on the J drive, while still very important, are older and were stored there really for occasional reference. 

In the last few days I've re-organised the files on the C drive so that many of the folders are now sub-folders of two main folders (if you see what i mean). I'm hoping and praying that this won't cause a problem.....

When this is sorted I would like to delete a lot of the back-ups. I can now see how important they are, but there are so many of them, and at up to 167MB each, they are taking up a huge amount of space on my HD, AND preventing my online back-up (Carbonite) from ever completing, as far as I can see. As a matter of interest, could the lrcat files be considered as "restore points", in effect?

Looking forward to your response. I'll be in all evening and will check back every so often....

cheers, jerry


----------



## Jim Wilde (May 6, 2010)

[quote author=jerry12953 link=topic=9715.msg66145#msg66145 date=1273174398]
Thanks very much for that incredibly detailed response and list of suggestions.

I think your alternative no2 is the one I would go for. I would have been quite happy to continue as I was had I been able to find the edits to the images on the J drive.
[/quote]

OK, if that's the one you want to go with we can start putting together a list of actions. Be aware though that of the 5'''+ pictures that you now have on the J drive, only 1571 of them will have metadata on that backup catalog....the other 4'''+ will not.

The other question I would ask: if you are now happy to have all these extra files in your catalog, why were they not already there? Does this point to an issue with your workflow that we perhaps need to address in due course.



> I'm glad to say that there appears to be no problem with the files or edits on the C drive.
> 
> In the last few days I've re-organised the files on the C drive so that many of the folders are now sub-folders of two main folders (if you see what i mean). I'm hoping and praying that this won't cause a problem.....



Me too! 

If you did this re-organisation from *within the Lightroom Folders Panel* then things should be fine. However, if you did it *outside* Lightroom then we almost certainly WILL have a problem.

Could I ask for a screenshot of the LH Panel of the Library module in the current master catalog. Make sure the Catalog and Folders Panels are both expanded, though for Folders I really just need Drives and Top Level Folders. Lower level sub-folders are not important at this stage.



> When this is sorted I would like to delete a lot of the back-ups. I can now see how important they are, but there are so many of them, and at up to 167MB each, they are taking up a huge amount of space on my HD, AND preventing my online back-up (Carbonite) from ever completing, as far as I can see. As a matter of interest, could the lrcat files be considered as "restore points", in effect?



Well, yes they could be considered Lightroom "restore points"....but not System Restore Points, and of course they do NOT provide a restore point for your actual images. Backing up of these remains the user's responsibility.

But we will address the backup strategy before we are finished....

One other question....there are a few rogue drives and associated 'missing files' in both catalogs. Are you aware of these....I'm presuming files associated with things like your CF card and DVD drive are merely past importing errors (Adding to Catalog instead of Copy and Add to Catalog), and thus it is safe to remove them? Think about that, but don't do anything yet until we have put some additional temporary backups in place.

Oh....where do you keep your catalogs? Local drive, J drive? Are they in a specific folder or in the root directory of the drive concerned?

Catch you later....off to see if my local "Peace Party" candidate swept all before him today!


----------



## jerry12953 (May 6, 2010)

Why do the other 4''' not have metadata? Good question.....possibly because they were pre-Lightroom images and I've never worked on them in LR. Some/many are tiffs/jpgs either from scans or of pre/post LR images. To be honest I don't know how come there are so many tiffs there, for example.

Yes I did do the re-organisation within LR!

I keep the catalogues on the C drive. To get them I just search for them....they are where they have always been.

I'm attaching a screenshot as requested, I hope.

Hope the election went well for you....I hope I'll be sleeping through all tonight's excitement.

jerry


----------



## Jim Wilde (May 7, 2010)

Jerry,

It's good that you did the folder organisation within Lightroom....but could you stop for the time being? Every change you make in one catalog will cause a problem in the other, and we have all the problems we need right now.

Could you give me a screenshot of the Lightroom folder hierarchy on your C drive using Windows Explorer....we are going to create a couple more catalogs when we start the recovery and I want to advise you where to put them. Thanks.

Looking at the last screenshot I see a lot of grey boxes and even more question marks on the thumbnails....I assume these will belong to some of the 41 images which are 'missing' from your Desktop? Can you confirm this? Do we need to be trying to locate them for Lightroom, or can we remove them from the catalog? I also see that there are 25 images which are still on your desktop and still in the catalog.....are they original raws? If they are still required in the catalog wouldn't it be a better idea to move them into one of your picture folders? 

Have a think about those questions, post me the screenshot, and I'll get back to you later.....

Cheers.


----------



## jerry12953 (May 7, 2010)

Hi,

The grey boxes and ?'s are indeed those "missing" on the desktop. They are almost all jpgs and tiffs which I believe I put on the desktop as a halfway house to burning or attaching to emails. Presumably the Raw files are still on the C or J drives. I'm not sure what to do with them in the meantime.

I'm hoping I've understood your request for another screenshot correctly and am attaching one.

I thought it was going to be easy from now on? 

jerry


----------



## Jim Wilde (May 7, 2010)

Jerry, I think the easiest thing to do regarding the missing files (given the transitory nature of their use of the 'desktop' folder) would be to remove them.....we can do that quite easily, but let's get cracking on the main procedure first.

Sorry, wrong choice of words...should have said we don't want to be making the problem any worse....but yes it should be relatively easy from now on....fingers crossed!

Whilst Denis is around (just seen his post in an 'old geezers' thread), I would ask him to cast a quick eye over the proposed method, which simply is:

1. Open '8'21'-1535 backup catalog, select all 1571 images on the J drive and 'Export as Catalog', creating a new interim catalog. Check box for Preview data, but NOT box for Export Negatives.
2. Close LR and restart using current master catalog. Take backup.
3. Import from Catalog, using the interim catalog from 2 as the source. Selct option to Replace Metadata and Preview Data only.
4. Wait for process to complete. Check some of those 1571 images to ensure metadata/edits have been restored.
5. Take another backup.
6. Do a little bit of a cleanup on those spurious missing files on desktop and other import volumes.
7. Later on, have a look at the backup strategy to see if any suggestions need to be made.

Assuming Denis gives this the OK, I'll format the relevant instructions for you....

Back later.


----------



## Denis Pagé (May 7, 2010)

As a catalog be it and original, a backed up or an exported one is still a catalog my question is: Why not import the '8'21'-1535 catalog directly from the master? There is no need for an intermediate I think. Or did I missed something? :


----------



## Jim Wilde (May 7, 2010)

Hi Denis....reason for the intermediate is that Jerry has been doing some reorganising of files on the C drive using the current catalog, so I would rather not risk complicating things by importing the whole of the backup catalog (which will now have an out of date C drive 'map')....hence the thought of using an interim step to isolate only the 1571 images on the J drive.

Happy to be guided by you if you think a direct import of backup into current catalog is a better or safer way to go....


----------



## Denis Pagé (May 7, 2010)

Ah! I see... By importing directly, you are not limited to the whole thing or nothing. You can select just what you want at import time. If you feel more confortable with a transitional export, then do so.


----------



## Jim Wilde (May 7, 2010)

My turn to say "Ah! I see..."! 

Can we select by volume or just by folder? If we can simply specify the J drive I agree a direct import makes more sense, but if we are limited to choosing folders an interim step might be easier as there aren't too many top level folders on Jerry's J drive....and there are a hell of a lot of picture folders to select!


----------



## Denis Pagé (May 7, 2010)

It won't show by drive letters. But I just made a test trying to import from a catalog having images on Local C:, LANs M: and CD reader D: (with CD in to avoid them being off line). Strange thing is that images on D: were not offered for the import! (Bug???).

May be simpler in you case of many folders to use that intermediary catalog...


----------



## Jim Wilde (May 7, 2010)

Thanks Denis, I appreciate your help and support.

I'll prepare some instructions for Jerry to have a look at later this evening. Do you want to see them first?


----------



## Denis Pagé (May 7, 2010)

Go ahead, I think you have enough info now. I mostly wanted to make you aware that direct import is in many (most?) cases an effective shortcut so you can use that knowledge in the future if it fits the bill...

P.S.: Will quit soon today...


----------



## jerry12953 (May 7, 2010)

Hi Jim,

I just got back to my computer. I see you've worked out a way forward but that there's no definitive instructions yet? I'll assume that I'm going to hear from you again before doing anything. For me it's too late at night for complicated software manouvres now anyway!

Just looking for a "yawn" smiley but can't find one........

jerry


----------



## Jim Wilde (May 8, 2010)

Jerry,

Aplogies for not getting back to you last night, unfortunately I had an accident with my right hand which meant a 4 hour wait in the A&E at Worthing Hospital (got home at 2 this morning). Not a major problem, but the hand is strapped up and I'm supposed to keep it in a sling for the next 4 or 5 days so lengthy posts with detailed instructions are not really possible for the time being (using the one-handed, one-finger typing method for this post! ).

So, can you wait a few more days? By all means carry on using your current catalog as normal, though suggest no file/folder reorganisation (*especially to the folders on thre J drive - that would be very bad!*). 

Sorry about this, I should be back to normal early next week....in the meantime if the delay causes you a problem just holler and I'm sure one of the experts here will step in and help.

Cheers.


----------



## jerry12953 (May 8, 2010)

No, there's no rush. Sorry to hear about the accident!

I'll try not to do anything silly in the meantime 

jerry


----------



## Victoria Bampton (May 11, 2010)

Aaaaaw Jim, look after yourself!


----------



## Jim Wilde (May 11, 2010)

Hi Jerry, as my hand is more or less back to normal thought I'd have a go at setting out those instructions for you:

*Stage 1 - Export the Metadata for the 1571 images on the J drive from the '8'21'-1535 Catalog*

1. Open the '8'21'-1535 catalog in Lightroom, and go to Grid View in the Library Module.
2. In the Folders Panel, single click on the entry for the Lacie Drive (J) (which should have the total of 1571 photographs). This will put all those 1571 pictures into the Grid.
3. Click on Edit&gt;Select All which should cause all 1571 pictures to become highlighted. Confirm this by looking at the Filmstrip and it should indicate that 1571 photos are indeed selected.
4. Click on File&gt;Export as Catalog and a browser dialog box (similar to my Capture1 screenshot attached) should appear. This is asking you to tell LR where to create the exported catalog and what to name it. If you already have a master folder of Lightroom Catalogs suggest you browse to that, then in the File Name box you could type something creative like Interim Export. 
5. *Note the two check-boxes at the bottom of the dialog box!* Make sure that "Export Negative Files" is NOT checked, and make sure the the "Include Available Previews" box IS checked.
6. Click on Save and Lightroom will proceed with the export. Likely to take a few minutes to copy the preview data (you should see a progress indicator top-left).
7. When the export has finished, you can close Lightroom and proceed to Stage 2. If you are interested you will be able to find, using Windows Explorer, the folder entitled Interim Export....inside which will be a file Interim Export.lrcat (the interim catalog) and a sub-folder called Interim Export Previews.lrdata (the preview data for those 1571 photos).

*If you encounter any issues with the above instructions, please advise before going on to later stages.*


*Stage 2 - Backup and Cleanup existing Master Catalog*

Note: This does NOT have to be done immediately after Stage 1.....it could be left for days if necessary, just do it when you are ready.

1. Start Lightroom using the current master catalog, *then take an immediate backup.* Make a note of the time and date of the backup as that will be our recovery point should anything go wrong later in the proceedings.
2. Folders Panel in the Library Module. Move cursor over the "dimmed' entry for Desktop that has the number 41 alongside (indicating 41 'missing' files). Making sure your mouse pointer is on the correct entry, right-click on it and select Remove. This will remove that entry from the Folders Panel.
3. Repeat for the other "dimmed" entry - DVD/RW Drive D - in the Folders Panel (which has 37 missing files).
4. At this point you should now have no more "dimmed" entries in the Folders Panel.

*Stage 3 - Import the Metadata from the Interim Catalog into the Master Catalog.*

Can be done at any time after Stage 2.

1. Open Lightroom using current master catalog.
2. Select File&gt;Import from Catalog.
3. A browser window will appear, asking you to identify the catalog for import. Browse to find the Interim Export.lrcat file (in the Interim Export folder), select it so that the name is in the File Name field and click on Choose.
4. Lightroom will then analyze the catalog being imported and will compare it to your open catalog. A dialog box will then appear which will look something like my Capture 2 screenshot attached.
5. In the New Photos section, the total should be ' and the options box will be greyed out.
6. In the Existing Photos box there should be a total of 1571. In the associated Replace options box, choose "Metadata and develop settings only". Click on Import and Lightroom will do it's thing. Note that it will only import the metadata/settings for changed photos only.
7. When it has finished you should be able to confirm that your 'Lost Edits' are restored, in which case take another backup and we are done!
8. One possible issue may be that the New Photos total will be more than ', in which case the Existing Photos total will be less that 1571....though the sum of New and Existing should still be 1571. This could arise IF you have subsequently removed some of those 1571 photos from your master catalog without deleting them. If this happens you can either choose an appropriate New Photos option and continue with the import, or you can cancel it pending a discussion about it (you can always restart the import later).

Obviously you will let me know if you run into any difficulties along the way!


----------



## jerry12953 (Apr 30, 2010)

I'm sure my problem is closely related to many others on here, but I can't relate the replies exactly to it.

I've lost all the edits to the images on my external hard drive. I can open them, but the history pane just says "Import '8/'2/1'".

That may well have been the date I moved everything over to a new PC, but I can't quite remember.....

To get access to the edits I can go to my lrcat backups and use one dating before '8/'2/1', but I can't, for example, add them to a collection started since them. And it's a pain, of course.

At one time the external HD had two letters "I" and "J" in the folders panel, but "I" has disappeared. 

How can I solve this one?


----------



## Jim Wilde (May 11, 2010)

[quote author=Victoria Bampton link=topic=9715.msg66332#msg66332 date=1273592429]
Aaaaaw Jim, look after yourself!
[/quote]

Thanks Victoria....everything back to normal!


----------



## jerry12953 (May 13, 2010)

Hi Jim,

I'm pleased to say that I've followed your instructions, as far as I can tell, with success! Thanks VERY much.

What I would like to do is delete a lot of my Lightroom back ups, especially those that I have created in the last couple of weeks while trying to solve this problem.

Do you have any suggestions?

jerry


----------



## Jim Wilde (May 13, 2010)

Hi Jerry,

Thanks for letting me know that the instructions seemed to work out! Good to know!

Re your backups, any chance you can open the Backups folder using Windows Explorer then take a screenshot. It would probably help us advise you which ones to keep and which ones can be deleted. One of the (slight) downsides, IMO, of the Lightroom Catalog backup process is that it is not possible to specify the number of backups to retain. Hence, if you are not diligent, LR will carry on creating backups according to the set frequency ad infinitum. And I agree, this can use up a lot of disk space over time, especially if you have it set to backup every time LR starts.

Other than the catalog backups taken by Lightroom itself, what other backup processes do you use? You backup your pictures folders, yes? Do you also take other backups of your Lightroom catalog folders? If you do, then in all probability you won't need to retain too many of the LR backups, just enough to protect against catalog corruption maybe.

Let's have a look at the Backups folder and we can take it from there.

Cheers.


----------



## jerry12953 (May 14, 2010)

Jim,

I've done a screenshot of the most up-to-date list of backups.....it goes on like this ad infinitum, so I can only copy the most recent.

I use Carbonite as my main backup, which is online, and the problem with having so many backups is that the queue for uploading them to Carbonite (which is quite slow) is never-ending. And that means that the image files don't get backed up immediately - or at all? - either. 

Maybe it would be a good idea to keep some critical backups round about '8'21' and 27'41'....just in case? And a load of the most recent ones?

I'll be guided by you.

cheers, jerry


----------



## Jim Wilde (May 14, 2010)

Jerry, 

Thanks for the screenshot (if you ever want to display more on the same screen, click on the small down arrow at the right-hand side of the "Organise" bar and select "List" from the various display options....this will give you a multi-column single-spaced list).

I really don't know too much about Carbonite as I don't use online backups myself, though I really don't understand why you should have so many catalog backups in an upload queue. One backup file is only the equivalent of 7 raw files from your 5DMkII, so if Carbonite is struggling with backups files, how on earth does it cope if you upload a shoot of many hundreds of raw files? Unless it is doing something totally dumb like backing up the complete backups *folder* each time you take another backup? Any Carbonite experts around to comment here?

In terms of your actual catalog backups on your hard drive, a couple of comments:

1. I would certainly agree that many (if not most) of these files could be deleted. I personally would only keep the '8'21'-1535 backup and say the previous 2 or 3, and then I would keep the last couple *before* you did that recovery procedure the other day, plus all backups since the recovery procedure. All the rest I would delete....but if you are nervous about deleting so many, copy them to a USB stick before deleting and delete from the stick in a couple of weeks when you are totally certain that you have no requirement to go back to one of them.

2. Backup frequency....there is probably no right or wrong answer here, the only correct frequency is the one that you are happiest with. Personally, I would not have the backup option set to "every time Lightroom starts", using the "once a day" option may well be enough. On my own system I actually have it set to "weekly", but that's only because I take my own copies of catalog, settings, and pictures folders (incrementally) and I'm disciplined enough to make two runs on the backup utility (to two separate drives) after each Lightroom session. I noticed on one day you had *eight* backups taken, which is certainly IMO a touch of overkill. Just have a think about the frequency and adjust as necessary.

Hope this helps...


----------



## jerry12953 (May 14, 2010)

I've been opening and closing LR like nobody's business while trying to sort this problem, going from one backup to another to check things, etc, which would not normally happen, so hopefully I can get a good system set up now.

If LR backed up when you closed it down (instead of when you opened up), you would know how much work you had done and whether you wanted a backup or not.

Carbonite is very slow to upload (at least on my connection), and deleting a lot of the backups already in the queue will help to ensure that the important stuff (new images etc) is backed up!

I'll have a go now.....


----------



## Jim Wilde (May 14, 2010)

OK, good luck!

In the Lightroom 3 beta, the backup option HAS been moved to the close down process....this has pleased some, but not all! I'm on the fence, as I said having Lightroom do the catalog backup is not a huge part of my workflow.


----------



## jerry12953 (May 14, 2010)

Having now deleted a large number of Lightroom backups, I can now see that the majority of the files awaiting upload to Carbonite are those files I moved around within Lightroom.

If Carbonite can't see them where it thought they were, it will wait 3' days and then delete them from your backup.

A very good reason for NOT moving your files around, even within LR, if you have an online back-up. It's so slow, I wonder if it will ever catch up.....

jerry


----------



## Bruce J (May 14, 2010)

[quote author=jerry12953 link=topic=9715.msg66487#msg66487 date=1273839386]
Having now deleted a large number of Lightroom backups, I can now see that the majority of the files awaiting upload to Carbonite are those files I moved around within Lightroom.
[/quote]

Jerry - just another quick comment for you to think about in your planning. LR backups zip quite well; mine wind up about 1'% of original size. I generally keep the most recent 4 or so in a zip file. I replace the oldest as new ones get created, on a once-a-week schedule. I also do a separate backup of the catalog file when I finish a significant session, so LR backups are just another, "just in case".


----------



## jerry12953 (May 16, 2010)

Thanks Bruce.....


----------

