# Imported images are ruined with random color patterns



## .jason (Dec 25, 2010)

I have LR 3.3 (camera raw 6.3) and I think around the time I upgraded from 3.2 my imported images display random areas of "digital garbage" on 40%-50% of the batch.  The initial import dialog window displays the thumbs normally, but when I import the files they are wack.

Doesn't matter if I'm in Library or Develop, 1:1, fit or full view.  I put the CF back into the 40D and it looks fine on the display.  Windows Explorer displays the thumb normally as well.

I thought the CF might be failing but all of the troubleshooting points to LR.  I found out from within LR, right click->edit in PhotoShop shows the badness.  Directly opening the CR2 in PS from the LR import shows the badness, but, if I open the CR2 from the CF directly in PS it looks normal.  To me it HAS to be LR botching the CR2 on import.

Lightroom Library Thumbnail






Lightroom Develop Panel





Viewing imported CR2 in PS (camera raw)





Viewing original CR2 from CF card in PS (camera raw)





Thanks in advance for any ideas on this!  (Merry XMAS, too!!!)


----------



## Brad Snyder (Dec 25, 2010)

Jason, welcome and Happy Holidays, too.

Those are classic signs of corrupted raw image files.  The fact that you see them OK in your camera or Windows Explorer has little bearing, those are showing you the embedded JPG, which is a small segment of the overall file.

You're going to have to take our word from quite a bit of experience, that it's extremely unlikely that Lr is 'causing' the problem. Lr is simply a very effective tool at 'recognizing' the problem.

This will boil down to some sort of  intermittent hardware failure in your image chain somewhere.
Camera, card, cable, reader, possibly memory or hard-drive (we argue about that a lot).

When you open the image in PS, is it opening up in the Adobe Camera Raw plugin?


----------



## .jason (Dec 25, 2010)

Hi Brad, yes in PS it was in camera raw 5.7.  The part I'm hung up on is why if I view the _imported _CR2 it has the markings but _direct from the CF_ into camera raw it's not visible?  I see LR uses a different (newer) camera raw version than CS4 does.  That's the best I could come up with.

Also, Explorer didn't show thumbs until I loaded the Canon codec.  Does this still mean I'm seeing the embedded jpg?


----------



## tzalman (Dec 25, 2010)

> Also, Explorer didn't show thumbs until I loaded the Canon codec. Does this still mean I'm seeing the embedded jpg?


Think about for a minute. You don't think Explorer is doing an on-the-fly conversion for the thumbnail, do you?


----------



## .jason (Dec 25, 2010)

tzalman said:


> Think about for a minute. You don't think Explorer is doing an on-the-fly conversion for the thumbnail, do you?


Wow, now that's helpful.  Thanks for the lesson TZALMAN.  I suppose if you reread the issue you'll find it wasn't the actual issue you addressed (or attemped to).  I was hoping for some insight, so I've disregarded your input.


----------



## .jason (Dec 25, 2010)

It seems like importing as DNG works for files that exhibit the issue when imported as CR2.

I've eliminated the reader, usb, CF... I think I know where this is heading...


----------



## Jim Wilde (Dec 25, 2010)

In situations like this you need to be trying to check out the various components of the image 'chain', which is what you are doing, although there are still quite a few things to check out.

Have you tried different CF cards? With what result?

Do you have more than one hard drive available? If you do I would try copying the CR2 files, that you know have exhibited the corruption, from the CF card to the second hard drive using Explorer. Then I would create a new clean temporary catalog and import those files from the second hard drive using the ADD function and see if the corruption is still evident. 

Do you have more than one PC available? If you have it would be helpful to try it with 'known' corrupt files.

If you typically use a card reader, have you tried copying files via a direct camera connection? With what result?


----------



## .jason (Dec 25, 2010)

I did try another known good CF, connected the USB direct to the body and imported.  I have not tried another PC or LR catalog.  I did notice the import looks proper as they begin appearing during import but they go wacky when the standard previews are rendered (just after the last file appears).

I'll try the new catalog next, then a different hard drive, and another PC if I have to go that far.  Thanks for the tip!


----------



## Jim Wilde (Dec 25, 2010)

The reason that the 'import looks proper' at first is because LR uses the embedded jpeg first, until it creates the standard preview from the actual CR2 raw file. All the indications are that somehow the raw file is being corrupted, the problem is finding out where in the chain that's happening.

To be honest, if I had a second PC available I would go straight to testing on that....a repeat of the problem on a second PC would eliminate at a stroke quite a lot of potential problem areas on the first PC (memory, hard drive, video driver, monitor profile, LR catalog, LR preferences, LR previews folder, Camera Raw Cache), and would narrow the problem down to camera, CF card, card reader (if you used the same one on both PCs).

Unfortunately as you can see, there are plenty of potential causes....you just need to be thorough and disciplined in your testing and hopefully you will eventually identify the culprit.


----------



## Brad Snyder (Dec 26, 2010)

As far as the CR2 codec and the JPG preview, I'm not sure what that shows you. I've never been able to get the da*n thing to work on Win7x64. If the results are snappy, I'm guessing it's still the JPG preview, because it does take a noticeable couple of seconds to crunch a CR2 raw render.

I do want to emphasize we're not a bunch of Lr apologists here, if this were a common problem attributable to Lr, we (at least I) would fess up. We've just never seen this unexplained by other (sometimes very hard to trace) problems.  Lr does not touch the original files at all, other than an OS file read, so there's just no opportunity for it to corrupt them. DNG complicates the issue a little bit, because there is a conversion step and a write, so there is an opportunity for corruption to arise, but again it not problem frequently attributed to Lr or the standalone converter.


----------



## .jason (Dec 26, 2010)

Loaded Lr on another PC and re-imported with the same exact setup (reader, usb cable, CF) and it worked perfecty... so it points to something with my primary PC.  Things are looking up!


----------



## Jim Wilde (Dec 26, 2010)

That's a good start!

Looking back over the thread, I see you tried to open a CR2 directly into PS (Camera Raw) reading the file on the CF card, and this was OK. However, the same action on the 'imported' file showed the corruption. As all the Lightroom import does is copy the files from card to hard disk, using the OS tools, the indications here are a problem with the hard drive. So, is it possible to try importing again into a different hard drive? You may need to use a temporary catalog as well as you don't really want to import the same pictures back into a catalog that still has them.


----------



## .jason (Dec 26, 2010)

New catalog to same HDD = fail
New catalog to new internal HDD = fail
New catalog to new NAS array = fail

Seems to be the PC (HP DL320 G5 server, Xeon x3210, 4G ECC RAM, Win 7 32bit)  This thing had worked for over a year without these issues.  No new software since this began either.  I don't get it!


----------



## Jim Wilde (Dec 26, 2010)

And the card reader you used on the alternate PC is the same as the one you're using on the failing PC?

I can't see why it would be the preferences file, but you could try trashing that....on Win7 you'll find the preferences file (Lightroom 3 Preferences.agprefs) in c:\users\_yourusername_\AppData\Roaming\Adobe\Lightroom\Preferences....simply rename the file or move it to a different folder then start Lightroom (it will create a new prefs file automatically) and repeat the test.

Assuming that has NO effect, it might be worth checking out your monitor profile and/or video card drivers. Do you calibrate your monitor? Have you recalibrated recently? Are your video card drivers up to date?


----------



## sizzlingbadger (Dec 26, 2010)

could it be a corrupted monitor profile ?


----------



## Brad Snyder (Dec 26, 2010)

Nik, what we're seeing there isn't the usual monitor profile symptom, but I don't think I'd rule it out at this point. That could explain some of the specificity of the problem. 

Jason, do you do a hardware calibration of the display on the problem system? If so how?  In the meantime, here's a workaround you can try which will help rule out that possible problem: http://www.lightroomqueen.com/blog/page/5/   scroll down until you see the 'Corrupted Monitor Profile' section.


----------



## .jason (Dec 26, 2010)

TNG said:


> And the card reader you used on the alternate PC is the same as the one you're using on the failing PC?
> Do you calibrate your monitor? Have you recalibrated recently? Are your video card drivers up to date?


Actually I've bypassed the reader and have been going direct to the 40D USB port (with a difference cable, and in a different group of USB ports.  The result was no different but in my quest to eliminate components it was an easy change.  I've not done a calibration quite some time.  Video card isn't 100% current either, so I can try that, too.



Brad Snyder said:


> Jason, do you do a hardware calibration of the display on the problem system? If so how?  In the meantime, here's a workaround you can try which will help rule out that possible problem: http://www.lightroomqueen.com/blog/page/5/   scroll down until you see the 'Corrupted Monitor Profile' section.


I'll give it a shot.  

BTW, thanks to all for all of the advice and assistance.  What a truly helpful and interested group of individuals I've run into here!


----------



## .jason (Dec 27, 2010)

No luck with the color profile or calibration either.  Another odd thing I missed before and just noticed was when opening in Ps as soon as Camera RAW displays the image it's jacked up for a split second and has a yellow exclamation in a triangle in the upper right corner, and then a second later it displays properly.  This is straight from the CF card, no import into Lr.

I know I mentioned this before but the Ps Camera RAW is 5.7 and Lr uses 6.3.  Is there anyway possible the newer handles CR2 differently?  Can I manually open the 6.3 executable?


----------



## Jim Wilde (Dec 27, 2010)

Jason, have you been able to try using a different hard drive?

It's unlikely to be the different versions of Camera Raw....there will be thousands of users (if not millions) using a 6.x version of Camera Raw within LR3.x, but an older version of Camera Raw with Photoshop. We would have heard if that 'mis-match' was the cause of this level of corruption in CR2 files. My 'test' machine uses LR3.3 (thus CR at 6.3) and Photoshop CS3 (Camera Raw 4.6), and has no such problems with my CR2 files.

To answer your last question, no it's not possible to have older versions of Photoshop (I assume you're using CS4?) utilise the latest version of Camera Raw.


----------



## .jason (Dec 27, 2010)

TNG said:


> Jason, have you been able to try using a different hard drive?.


Yup.  Yesterday.  No bueno.


----------



## Jim Wilde (Dec 27, 2010)

Oh dear....well I'm about out of ideas. The only thing left on my list would be a possible memory problem. Have you tried running a memory test?

Is your 4GB of RAM in two sticks? If so you could maybe try pulling each one alternately to see if anything changes.....


----------



## Victoria Bampton (Dec 28, 2010)

Memory would be my next port of call too - I've heard of similar problems from memory going bad.


----------



## Brad Snyder (Dec 28, 2010)

Victoria Bampton said:


> Memory would be my next port of call too - I've heard of similar problems from memory going bad.


 
On the other hand, we've had extended discussions here, whether such a thing is likely, or even possible, in the absence of other system symptoms.  

I'm with Vic and Jim, I've seen enough weird cr*p happen to not rule out anything. If you're even just a little bit handy, this one is easy enough to test.


----------



## ukbrown (Dec 29, 2010)

> As far as the CR2 codec and the JPG preview, I'm not sure what that  shows you. I've never been able to get the da*n thing to work on  Win7x64. If the results are snappy, I'm guessing it's still the JPG  preview, because it does take a noticeable couple of seconds to crunch a  CR2 raw render.



@brad, thehttp://www.ardfry.com/cr2-codec/  codecs work very well in previewing raw files.

Do CR2 raw files actually have a jpg preview built in?


----------



## ukbrown (Dec 29, 2010)

As one of the proponents of the bad memory causes crashes not arbitary wrong images on the screen, I would look at your graphics card.  You also made a statement about if you converted to DNG the images seemed to be OK when the RAW ones were not, no one followed that up, did you find the reason for that, I have re-read the thread and cannot see it.  In simialr cases doing file compares between the various files that are mvoed about can show if the files have changed in any way.  I know on my computer LR uses the graphics card the most, if you have faulty memory in that it won't affect the running of the PC, just what you see, same for the graphics driver most times.

Just to cover the theory of main RAM causing crashes see below, it's only a theory, but I think it holds water, if someone can spot any holes then i will let it sink

If your main memory which holds programs, data and graphic information has a tendency to change and you blame it for dodgy pictures it follows that the programs that are stored in memory will also change and therfore the instructions that are presented to the CPU will also be scrambled, it is the same memory and except in the case of shared memory on board graphics, you have no control of what gets put where, the OS does it.  Therefore LR or any other program will start to issue a series of arbitrary instructions to your processor, your processor will execute them.  At the very least you will be experiencing unpredictable program behaviour and at the worst, blue screen crashes as the OS protects itself from rogue code.


----------



## .jason (Dec 25, 2010)

I have LR 3.3 (camera raw 6.3) and I think around the time I upgraded from 3.2 my imported images display random areas of "digital garbage" on 40%-50% of the batch.  The initial import dialog window displays the thumbs normally, but when I import the files they are wack.

Doesn't matter if I'm in Library or Develop, 1:1, fit or full view.  I put the CF back into the 40D and it looks fine on the display.  Windows Explorer displays the thumb normally as well.

I thought the CF might be failing but all of the troubleshooting points to LR.  I found out from within LR, right click->edit in PhotoShop shows the badness.  Directly opening the CR2 in PS from the LR import shows the badness, but, if I open the CR2 from the CF directly in PS it looks normal.  To me it HAS to be LR botching the CR2 on import.

Lightroom Library Thumbnail






Lightroom Develop Panel





Viewing imported CR2 in PS (camera raw)





Viewing original CR2 from CF card in PS (camera raw)





Thanks in advance for any ideas on this!  (Merry XMAS, too!!!)


----------



## Brad Snyder (Dec 30, 2010)

ukbrown, thanks. I've used the ardfry, and I'm currently using http://www.fastpictureviewer.com/  . I wasn't very clear, but it was specifically the Canon codec, that annoys me. It's hard to download and install, and the last time I checked wouldn't run in 64 bit (or some similar complication).

I'm pretty sure modern vintage CR2s have embedded JPG previews. Years ago, CRWs had .THM sidecars which were basically JPG previews.


----------



## b_gossweiler (Dec 30, 2010)

Brad Snyder said:


> ... I wasn't very clear, but it was specifically the Canon codec, that annoys me. It's hard to download and install, and the last time I checked wouldn't run in 64 bit (or some similar complication).


 
And it is still not supported under 64Bit, just tried to install it during my Win7 upgrade adventure.

Beat


----------



## Jim Wilde (Dec 30, 2010)

A couple more things to maybe try:

1. Trash the ACR Cache...looking at post #18 in this thread, I was interested to hear of the 'temporary' corruption when opening the CR2 directly in PS from the CF card, which then cleared itself. Other that more global system-wide elements such as video-card, memory, or indeed OS, the only common factors between Lightroom and PS in this instance are the USB hardware and the ACR Cache. Checking out the Cache is easy to do, simply find it and rename it....Lighroom will then recreate the file on next startup (you could use the option to Purge, but a temporary rename lets you go back to it if it turns out NOT to be the problem). To locate the Cache file, within Lightroom go to Edit>Preferences>File Handling Tab.....the existing path is down near the bottom of the page, close LR, locate the file in Explorer then rename it (OldCache?), restart LR and check again for the problem.

2. If that has no effect, try circumventing your USB controller on the PC in question....I notice you have a NAS, can you copy from the CF card on the OTHER PC directly to the NAS, then on the 'problem' PC try to import into Lightroom (use Add to keep the files on the NAS) and see the results?

Would be interested in knowing the outcome....


----------



## 6sawins (Mar 30, 2011)

I am now having the same problem.  I read the threads and have tried most of the suggestions, and I agree it does not make sense.  I have examples from a canon 40D and 7D; and at least two different CF cards.  When I view the thumbnail in the 3.3 import source screen it displays find.  When import is complete the random color patterns appear.  I tried a new catalog, and on a different HD; and existing catalog same HD and again on a different HD.  I also notice with a free raw viewer the raw cr2 on HD looks fine, if I import and convert to .dng then even the free raw viewer shows the random color patterns.  So whatever LR is seeing in the file is duplicated when creating .dng.  Any progress on this issue?


----------



## Mark Sirota (Mar 30, 2011)

6sawins said:


> When I view the thumbnail in the 3.3 import source screen it displays find.  When import is complete the random color patterns appear.  I tried a new catalog, and on a different HD; and existing catalog same HD and again on a different HD.  I also notice with a free raw viewer the raw cr2 on HD looks fine, if I import and convert to .dng then even the free raw viewer shows the random color patterns.  So whatever LR is seeing in the file is duplicated when creating .dng.



Welcome to Lightroom Forums.

This part all makes sense.  What's happened is that the raw data within the file is corrupt, but the JPEG preview embedded inside the raw file is okay.  LR's Import window and the free raw viewer are both showing you the embedded preview.  But once LR tries to render its own preview from the raw data, it goes bad.  This also happens when you convert to DNG.

How are you reading those cards?  Are you connecting the camera with a cable, and if so, did you use the same cable for both cameras?  Or are you using a card reader, and if so, did you use the same card reader for both cards?  It's not uncommon to have a reader or a cable go bad.


----------



## Jason (Apr 1, 2011)

Hmm.. Well, this seems to be picky, doesn't it?  So a new computer works just fine...  Have you tried diagnostics on the computer itself, then?  Starting with memory (probably not), moving on to the disk via disk controller?

This type of thing is normally a transfer problem, from what I've seen, but you've addressed many of the common problems.  But it does seem isolated to your particular computer...  Have you updated your motherboard drivers?  Are they up-to-date?  Worth a check, anyway.  Then I would try some hardware diagnostics, to help eliminate a few things.

Cheers!


----------

